nim Composition Space Optimization is a high-performance tool leveraging metaprogramming to implement several methods for selecting components (data dimensions) in compositional datasets, as to optimize the data availability and density for applications such as machine learning.
A Python tool to identify and remove similar-looking images from a dataset. Utilizes image preprocessing and hashing techniques for efficient comparison.
⚡ Vectro: Lightning-fast embedding quantization. Hit 12M+ vec/s throughput (4.85× faster than FAISS C++) while drastically cutting memory footprint for vector databases and local AI research.
Highlighting expertise in data migration, data normalization and standardization, this project demonstrates successful data transfer from Snowflake to Databricks. It emphasizes optimized data flow and enhanced accessibility through standardization, showcasing a commitment to ethical data practices.
Morphia is a powerful library designed for transforming complex JSON structures into compact, index-based arrays and back. Ideal for efficient data serialization and deserialization, Morphia simplifies the handling of nested and dynamic data by flattening JSON objects into arrays and reconstructing them with precision.
interface to ADAO (Data Assimilation and Optimization features in Python ) in SALOME
An award-winning solution proposal for optimizing vertiport site selection in Urban Air Mobility (UAM) using multi-source data analysis and platform logic design. Recognized at the 9th Hyundai Motors University Case Competition.
Contains my project that analyzes air quality sensor data to determine if the NO (Nitric Oxide) sensor in N. Mai, Los Angeles, CA can be removed without affecting data accuracy.
UI website for the ESAT Team 5A2 Smart Grid Data Project
Lobster data analysis and modelling using Python
This explores data optimization using PySpark, covering big data concepts, pipelines, machine learning, and data transformation techniques. The goal is to enhance data processing efficiency and model performance.
Official Tensorflow implementation for ActiveTCR: Active Learning Framwork for Cost-Effective TCR-Epitope Binding Affinity Prediction
A CLI-based, open-source AI memory system that uses an existing local LLM to analyze and permanently store information from text-based sources such as PDFs, XLSX, DOCX, TXT, CSV, and JSON. The system converts data into an efficient searchable representation, stores it locally, and retrieves relevant information when answering questions
Compress and minify CSS stylesheets with comment stripping and token reduction offline.
fast dataset merging and deduplication tool
Production Delta Lake Medallion architecture. Orchestrates Bronze→Silver→Gold pipeline with PII masking, deduplication, MERGE/Upsert, OPTIMIZE/VACUUM, and Airflow DAGs.
Fullstack Developer for NEV/MOSY | Specializing in Extreme Data Optimization & Minification (PI-API Developer). TypeScript/JavaScript Architect.
This project analyzes historical weather data to identify patterns and predict future weather conditions, focusing on extreme events and temperature trends across Europe.
Add a description, image, and links to the data-optimization topic page so that developers can more easily learn about it.
To associate your repository with the data-optimization topic, visit your repo's landing page and select "manage topics."