omxplorer by omxlabs Live now

Ask your data in a sentence.

Robotics viewers replay one recording. omxplorer starts from a question across the whole fleet, then shows you the moment. A million rows answer in under a millisecond on your own GPU, and every button is a tool your AI agent can call.

▶ Watch the 2-minute slidesThe robot storyGitHub
Linux runsMac runs on CI MacsWeb runs
omxplorer · Linux · why did the robot drop it?

Why did the robot drop it? 63,344 real robot episodes from 13 labs, their video and 16.8M telemetry rows, linked by key and time. A question in words over every row in about 40 ms.
↳ DROID (CC-BY-4.0) · omxcode docs/goals/47-report.md

0.6msa chart over 1 million rows, grouped and sortedRTX 3090 · chart 0.5–0.7 ms · omxcode README
0.8msa frame, scrolling a 1M-row gridp95 · omxcode docs/goals/26-report.md
30msa filter change over 100M rows, from input to pixelsmedian · Rust + Polars 37.0, JS + DuckDB 104.9 · measured at 856a7e9, before the release build · docs/data-benchmark.md
0.2sto open a 100-million-row file, streamedfirst rows on screen; fully in memory from Parquet 1.8 s · goals 22 and 30

NVIDIA RTX 3090 · the repo's own checks re-run each number · honest edge: a 100M-row median chart takes about 0.7 s (0.77 s in goal 30)

Live · real data

Why did the robot drop it? One sheet, its video, its telemetry, its paper.

The whole fleet is a spreadsheet you touch: 63,344 real episodes from 13 labs (DROID, CC-BY-4.0), pivots and charts from a sentence. Ask in words over all 16.8M telemetry rows in about 40 ms, with no Python and no server, then tap an answer: its video and telemetry open on one cursor, its documents linked.

A failed episode's video beside four lanes of its telemetry on one cursor: the gripper opens over the box's edge and the shirt falls
The drop, on one cursor.F00118: the gripper pinches the shirt at 5.3 s, opens over the box's edge at 12.7 s, and the shirt falls outside.video + 244 telemetry rows, opened by links alone · goal 47
The DROID fleet as a sheet: 63,344 episodes, one row each
The fleet is the sheet.63,344 episodes, one row each: lab, robot, task, outcome, clip.DROID, CC-BY-4.0
A chart of average success by lab
Success by lab.From PennPAL's 95 % down to RAD's 61 %, typed as a sentence.chart · goal 47
The Links panel: the episode key, its video file, the clock and the paper, each confirmed
Nothing joins until you link it.The episode key, its video file, the clock, its paper.explicit links · goal 47
A question in words answered by a join over 16.8 million telemetry rows
Asked in words."Failures where the gripper closed, let go and closed again": 2,997 episodes, about 40 ms.JOIN LINKED over 16.8M rows
The DROID paper as a document card, the passage 'recorded at 15Hz' marked
The paper, by its key.Every row names the DROID paper; its "recorded at 15Hz" passage is marked, read from the PDF's own text.arXiv 2403.12945, CC-BY-4.0
Export pack: a PDF per picked failure with the frame at the release and the telemetry there
A pack per failure.A PDF with the frame where it let go, the telemetry there and the paper's page, plus a 6 s clip.Export pack · goal 47

What the data doesn't support: DROID has no force or torque, so "dropped" is read from the gripper and the video; the per-episode gripper features don't predict failure across labs, so they're filters, not a detector · omxcode docs/goals/47-report.md

Where omxplorer is different

Not a replay. An investigation.

Robotics viewers replay a recording. omxplorer starts from a question across the whole fleet, then shows you the moment.

4 · Links you confirm by hand

Key, time, place: drag, confirm, done.

No schema written in code. Drag a column onto a card and say yes. Nothing joins until you link it, and a question uses only the links you made, so every answer can show where it came from.

1

The fleet is a spreadsheet you touch.

Pivots, charts from a sentence, formulas and edits, over every run.

2

Question → moment, in one motion.

Ask in words over 16.8M rows (~40 ms), tap an answer row, and its video and telemetry open on the same cursor.

3

Documents in the same join.

A manual or paper linked by key, with the passage marked.

5

No server for the analytics.

The query runs in the app's process, on your GPU.

16.8M telemetry rows · 37–42 ms in the check's runs · links, documents and the pack: omxcode docs/goals/47-report.md

What it is

Five things nothing else does together.

💬

The title is the chart.

“Sum of revenue by product, top 5, as bars.” Every coloured word is a setting. Tap one, or type “median”, and the bars move to the new answer.

🤖

Every button is a tool.

omxplorer is an MCP server from the first line. An agent calls what a person can tap, and nothing more.

⚡

Your GPU does the work.

Group-bys, medians and charts run as GPU kernels in omxcore, linked in like SQLite.

📂

100M rows, no warehouse.

Open CSV and Parquet straight from disk. Big files stream in, so the first rows show in 0.2 s.

🖥️

Native everywhere.

Linux and the web run today, and the Mac runs on CI Macs (GitHub's macOS runners), from the same Mojo. Phones are next.

Ask in a sentence. Change it with a tap.

Sum of revenue by product, largest first, top 5, as bars

omxplorer · orders_1m.csv · 1,000,000 rows
  • ⏱️ It shows its work.Every answer says how long it took and where it ran.
  • 🙋 It's honest.Ask for something it can't do, and it names what it skipped.
  • 🤫 It's quiet.One accent, short motion, nothing in your way.

Every button is a tool.

Most apps meet AI agents through screenshots and guesswork. In omxplorer, every control your hands can reach, an agent can call. A disabled button is no tool.

A real session · no screenshots
→ choose-a-file  {"path": "orders_1m.csv"}
→ create-table
→ insert-chart
→ ask  {"text": "revenue by product, largest first, top 5"}
← Now: chart0: 0.6 ms
Give it to Claude Code in one command: scripts/agent.sh omxplorer Read the agent docs
the session is from the omxcode README · docs/agent.md
LiveIn progress

Video becomes rows.

Open a video and it turns into a table you can ask, like any CSV: about 400 frames a second, on your GPU. Ask for a reel, and the rows cut it.

omxplorer · warehouse.mp4 · 1,140 frames
The warehouse, a real recording driven by omxplorer's agent tools: open, Analyze, the detections table, a reel of damaged pallets (faces blurred, shipment ID burned in). Then the video card's Edit: Cut it makes 3 clips from the rows.
omxplorer · two drone flights · telemetry
The drone, in omxplorer, driven by its agent tools: the flight with its telemetry (the chart's cursor and the map's dot follow the video), Analyze, then two flights compared: the assets that got worse, and the next flight plan.

Built and running

  • Video becomes rows on the GPU, about 400 frames a second: every pallet, code and face, frame by frame.
  • The reel: the frames where a column is true, faces blurred, the shipment ID burned in.
  • SQL joins video rows with a CSV: the delivered shipments whose label never shows is one query.
  • The drone: two inspection flights become a table of every asset and defect on the map; which got worse since last time is one tap, and the answer becomes the next flight plan.
  • Every step is an agent tool, both clips included.

Not yet

  • The footage is generated (so we know the truth of every frame). Real-world footage hasn't been tried.
  • The models (OWLv2, SAM 2.1) run on Linux with NVIDIA only.
  • The defect severity scale is calibrated to how our footage was drawn. On real footage it would need calibrating again.
  • Models inside omxplorer. Find more like this (OWLv2, from one tapped example) runs in process in the framework's tests: recall 1.000, precision 0.886 on our footage. omxplorer itself still uses classic vision.
  • Audio.
LiveIn progress

One question, three kinds of evidence.

The dock camera, the shipments sheet and the signed bills of lading answer one question: which damaged pallets does the carrier owe for? You link them yourself, then pick any answer to see its proof.

omxplorer · dock.mp4 · shipments.csv · 17 bills of lading
The video, the sheet and the paperwork land. Analyze turns the video into rows; Read finds the bills' fields. A drag joins shipment_id to the video ("Link these?"), and another joins it to the bills. Hold a card to see what it's linked to. Ask in words, with the SQL beside it. Pick an answer and the video jumps to the damage while the bill marks "Exceptions: None". Then Export pack: a PDF and a clip for each shipment.

Built and running

  • Explicit links: drag a column onto a card and confirm. Nothing is joined until you say so, and JOIN LINKED uses only the links you made.
  • Documents become rows: a folder of PDFs becomes one card. You name the labels printed on the form and the fields land as a table. The OCR (docTR) runs locally, inside the app, on your GPU.
  • Every answer shows its proof: pick a row and the video seeks to that frame while the page highlights the field.
  • Claim pack: a PDF and a blurred, captioned clip for each shipment.
  • Reproducible: real functionality on generated data. One script rebuilds the same files, byte for byte, and every step in the clip is an agent tool.

Not yet

  • Real warehouse footage. The dock video is generated, so every frame's truth is known.
  • Scanned or handwritten documents. Our bills are clean PDFs. Scans are only scored on a public research set, never shipped.
  • Uploading a folder on the web. Folders open in the desktop app and through agent tools.
  • OCR on Linux with NVIDIA only, for now.
How it works

One row opens everything linked to it.

Press Play to follow a failure from the sheet to the moment the shirt falls, or click any block to read what it does.

Why did it drop?Open full screen ›

The fleet sheet, its video, its 16.8M telemetry rows and the DROID paper, joined only by the links you confirm; omxcore answers questions over them in the same process.

Roadmap

Where it's going. From the fleet to the moment.

Data and robotics first, then every screen. Each step ships when its checks pass.

Now

Data, video and documents, linked

Sheets, pivots, charts from a sentence over 100M-row files; video becomes rows; the claim; the robot story. Linux, the web, CI Macs.

runs today
Next

More of a recording

3D scenes, MCAP and ROS bags, more cameras per episode, beside the fleet sheet.

next
Next

In your browser, models in the app

omxplorer live on omxlabs.tech (hosting not set up yet); OWLv2 and SAM 2.1 inside omxplorer; Ask answered by omxconsole on your GPU.

next
Then

Teams, audio, real footage

Share an investigation with a team; audio becomes rows; real-world footage at scale; Apple GPUs on real Macs (CI Macs today).

planned
The slides · 18 · about two minutes

The why, the how, and the code.

Click the slides, then use the arrow keys. Code steps line by line; the architecture moves as a tap runs.

omxplorer · the slidesFull screen ↗
Under omxplorer

Powered by three technologies.

See a million rows answer in under a millisecond.

▶ Watch the slidesRun it from GitHub