The Internet, 22 June 2026

I'm happy to announce pyacid v0.6.1 — 26 changes over 162 commits since the last note; highlights below, see the website for the rest.

WARNING: acid is alpha software, developing rapidly — the API will change between releases. Pin a version (pip install pyacid==0.6.1) for anything you rely on, and watch these emails for what changed.

Headline new features:

1. Import your own catalogs. acid can now build catalogs, not just read them — acid import turns your files (Parquet, CSV/TSV, FITS, Arrow) or a Rubin Butler dataset into a queryable, margin-cached HATS catalog.

# from a pile of survey files...
acid import fits /data/legacy/tractor/*.fits --out legacy_dr10 --ra ra --dec dec

# ...or straight from a Rubin Butler — including remote DP1, no LSST stack
acid import butler --repo dp1 --collection LSSTComCam/DP1 --dataset object \
    --out dp1_object --ra coord_ra --dec coord_dec

Until now acid could only query catalogs someone else had already converted to HATS; now you bring your own and crossmatch them like any other — laptop or cluster, in memory for small inputs and out-of-core for large ones. The Butler path needs no LSST stack: a remote repo like Rubin DP1 works from a laptop with just pip install pyacid and an access token.

A caveat in the honest spirit of alpha: acid import isn't fully auto-tuned yet. For very large catalogs (10B+ rows) or memory-tight machines, you may need to set --workers and the memory options by hand to avoid running out of RAM — acid import peek (below) now reports a per-row byte estimate to help you size them.

2. Peek before you import. acid import peek shows what's actually in your inputs — columns, row counts, candidate RA/Dec, the files that match — reading only metadata and a small sample, even when the data lives on a remote host. Point it at a Rubin Butler and it's a fast way to explore a repo's collections and datasets without running a query.

# what's in these files? (remote hosts included — reads metadata only)
acid import peek 's3://my-survey/dr1/*.parquet'

# browse a Rubin Butler repo — collections, datasets, schemas — fast
acid import peek butler --repo dp1 --collection LSSTComCam/DP1

Check column names, find your RA/Dec, and confirm a glob matches what you expect before kicking off a big import — and explore a Butler (remote DP1 included) without writing Butler queries. peek now also reports each column's byte size and a per-row total, so you can size --workers / --ram-budget before you commit.

3. Units preserved end-to-end. Every catalog acid writes — from an import or a save — now keeps its columns' units, descriptions, and UCDs, wherever the source provides them. They stay on disk, reopen with the catalog, and flow through your queries into .to_astropy().

import acid, astropy.units as u
from astropy.table import Table

targets = Table({"ra": [180.0, 181.0], "dec": [2.0, 2.1], "vlos": [42, 55]*u.km/u.s})
acid.open(targets, ra="ra", dec="dec").save("my_targets")  # units stored in the catalog
acid.open("my_targets").to_astropy()["vlos"].unit          # km / s — straight off disk

Your imported and saved catalogs are now self-describing — units live with the data instead of in a README you have to remember, and reopen with the catalog for whoever queries it next.

Upgrade: pip install -U pyacid · Full changelog →

Also new:

• Every import ends with a color sky map of the new catalog — where it landed, how dense, and how much sky it covers — so you can eyeball that it worked.
• Catalog.skymap() returns a HEALPix density map of any query result.

(v0.6.1 also smoothed out remote-Butler import — fixing repo-index resolution over http/s3, and surfacing server-side query errors instead of swallowing them.)

Thanks for your time — as always, see the website and documentation for the details. If you hit any problems or have feature ideas, open an issue at github.com/mjuric/acid-issues.

— Mario & his AI buddies