Description
vaex is an agent skill for processing large tabular scientific datasets with Vaex on a single machine. It is for data that is larger than memory, especially for repeated reductions and histograms over local Vaex HDF5 or Arrow files. Vaex uses expressions and virtual columns, which defer computation until it is needed, and reductions normally run straight away. It also supports filtered views, streamed statistics, binned visualisations and conversion between file formats. The skill is clear that out-of-core storage does not make every operation memory-bounded. Sorting, joins, large group dictionaries, materialising columns and many estimator fits still need substantial RAM. The installation guide says to use a separate environment, because the repository's default Python version is newer than this Vaex release supports. Use it when a table is too large for pandas and your work is mostly summaries, filters and plots over the data. It is part of the k-dense-ai scientific agent skills repository on GitHub, which has about 49,600 stars. The listing shows it as free, and it has about 1,800 installs.…
Highlights
- Out-of-core tabular processing
- Lazy expressions and virtual columns
- Streamed statistics and histograms
- Filtered views and file conversion
- HDF5 and Arrow file support
Ratings & Reviews
No ratings yet
Be the first to rate vaex.
People also visit
vaex FAQ
When should I use Vaex?
For data larger than memory on one machine, especially repeated reductions and histograms over HDF5 or Arrow files.
Does Vaex keep every operation within memory?
No. The skill says sorting, joins, large group dictionaries and many estimator fits still need substantial RAM.
What is lazy about it?
Expressions and virtual columns defer computation, while reductions usually run at once.
Which environment should I use?
A separate one, since the repository's default Python is newer than this release supports.
How do I install it?
Run npx skills add https://github.com/k-dense-ai/scientific-agent-skills --skill vaex.
Is it checked for security?
Its security audits show a warning from Gen Agent Trust Hub, and passes from Socket and Snyk.
What are the best vaex alternatives?
Popular alternatives to vaex include bigquery-bigframes, pandas-pro and data-scraper-agent. See all vaex alternatives