# IndieHackers collector

A native Odin CLI that collects product search results from IndieHackers' Algolia index and exports a UTF-8 CSV. It preserves the flags, filters, defaults, and six columns of [Rinkoff/indiehackers-collector](https://github.com/Rinkoff/indiehackers-collector/blob/5d0fab2477f7db37270464630452d701a7ae0c3b/collector.py), reviewed at commit `5d0fab2477f7db37270464630452d701a7ae0c3b`. The executable uses libcurl directly, a strict decoder around Odin's JSON tokenizer, and the standard CSV encoder. It has no Python runtime or package dependencies.

The [visual project guide](https://indiehacker.plosca.ru/) explains the request, eight source modules, CLI filters, CSV publication and adversarial review through interactive diagrams. Its examples use synthetic data. The static guide and its Caddy deployment live in [guide/](guide/README.md); they add no runtime dependency to the collector.

Build with the official monthly [Odin dev-2026-10 release](https://github.com/odin-lang/Odin/releases/tag/dev-2026-10), the latest release marked non-prerelease when checked on 2026-10-17. Odin does not yet publish a `1.x` stable series; its monthly release tags include `dev`. The official Linux archive reports `dev-2026-10-nightly:84bc3fc` internally. This is the version tested here.

Linux is the supported platform; x86-64 is verified here. You need Clang on `PATH`, the system libcurl and zlib libraries, and a working CA certificate store. The compiler's `vendor:curl` package supplies the bindings; there is no dependency manager or vendored library build. See [Odin's installation instructions](https://odin-lang.org/docs/install/) for compiler setup on another machine. On Arch Linux the relevant system packages are `clang`, `curl`, `zlib`, and `ca-certificates`.

```sh
odin version
make
./build/collector --help
```

Create your local configuration and replace the placeholders with the search-only credentials observed in your browser's normal products-page requests:

```sh
cp .env.example .env
```

```dotenv
ALGOLIA_APP_ID=your_app_id_here
ALGOLIA_API_KEY=your_search_only_api_key_here
ALGOLIA_INDEX=products
```

The collector reads `.env` from the current working directory. It must be a regular UTF-8 file no larger than 64 KiB; symlinks to regular configuration files work. Exported environment variables take precedence, including an explicitly empty value. `ALGOLIA_INDEX` defaults to `products` when unset. The dotenv reader accepts one-line assignments, optional `export`, single or double quotes, comments, UTF-8 BOM, and CRLF. Quoted values preserve hashes; unquoted inline comments start with whitespace followed by `#`. Double-quoted values support `\n`, `\r`, `\t`, `\"`, and `\\`. It does not evaluate shell commands, expand `${VARIABLES}`, search parent directories, or accept multiline values. Unknown variables are ignored. `.env`, generated CSVs in the repository root, staging files, and build artifacts are ignored by Git.

Run from the directory containing your configuration:

```sh
# Defaults: founder codes, minimum $1/month, at most 5 pages.
./build/collector

# Revenue is inclusive: at least $1,000/month.
./build/collector --revenue 1000 --pages 3 --output high_revenue.csv

# Remove the founder-code filter, including both coders and non-coders.
./build/collector --no-code --pages 1

# Disable both filters.
./build/collector --no-code --revenue 0
```

| Option | Default | Behavior |
| --- | --- | --- |
| `--revenue INTEGER` | `1` | Minimum monthly revenue, inclusive; zero or negative disables this filter |
| `--no-code` | Off | Remove `_tags:founders-code-yes` from the query |
| `--pages INTEGER` | `5` | Maximum pages, requesting 100 hits per page; zero skips collection |
| `--output FILE` | `products.csv` | CSV destination; its parent directory must exist |
| `--base-url URL` | `https://{APP_ID}-dsn.algolia.net` | Override the API origin for local testing; no path, query, or userinfo |
| `-h`, `--help` | | Display usage without needing credentials |

Options that take values accept both `--pages 3` and `--pages=3`. Integers are signed decimal values within the 64-bit range; page counts must be nonnegative. The original four flags and their defaults are preserved. `--base-url` is the single additional option and sends the configured authentication headers to the explicitly chosen server.

Each request uses `POST /1/indexes/{index}/query`, an empty search query, Algolia authentication headers, and the same filter expression as the reference. Index names are URL-encoded; `.` and `..` are rejected. The collector reuses one HTTP client, waits one second between pages, and stops at the requested page limit, an empty result page, or `nbPages`. Logs distinguish the requested limit from the last available page. Algolia controls the accessible result window; this tool cannot guarantee access to every product in the index. See the [Algolia search API](https://www.algolia.com/doc/rest-api/search/search-single-index) for the request and pagination contract.

The CSV columns, in order, are:

```csv
Name,Tagline,Monthly Revenue ($),IndieHackers URL,Website,Status
```

Missing fields use the reference defaults: `N/A` for name, `0` for revenue, `unknown` for status, and empty strings for the other fields. URLs use `https://www.indiehackers.com/product/{objectID}`, with the ID encoded as one path segment. Explicit JSON nulls produce empty fields, including an empty object ID in the URL. Numeric values are formatted individually without pandas' whole-column type coercion. CSV quoting handles commas, quotes, line breaks, and Unicode; line endings are LF. The CSV preserves source text, including formula-like strings that spreadsheet applications may interpret.

Several failure behaviors deliberately improve on the Python script. Request, configuration, JSON/schema, and file errors exit with status `1`; invalid CLI options exit with `2`. Successful runs, including empty results, exit with `0`. SIGINT/SIGTERM cancel collection and return `130`/`143`. Search responses require HTTP `200`, a `hits` array of at most 100 items, and a nonnegative integer `nbPages`; if `page` is present it must match the request. Repeated nonempty product IDs within or across pages abort the collection instead of silently duplicating rows. Missing/null IDs retain the reference's empty-URL-suffix behavior.

The decoder rejects malformed JSON, duplicate object keys (including empty or escaped aliases), invalid UTF-8, and unpaired Unicode surrogates. Exported product fields must be scalar values or null. Integers must fit signed 64 bits and floating-point values must be finite. Each response is limited to 16 MiB, 128 nesting levels, and 200,000 tokens. HTTP redirects are rejected, TLS certificate verification stays enabled, and each request has a total 15-second deadline. Errors do not print API keys or remote response bodies. There are no application retries.

Rows stream through a checked 16 KiB buffer into a private staging file, with page allocations released after each response. New exports use mode `0600`; replacements preserve existing permission bits. The collector rejects symlinks and special destination files. Export operations use an open directory descriptor, so replacing a path cannot redirect cleanup. A nonblocking kernel lock allows one collector per output directory and leaves no lock files. Destination identity and timestamps are checked before publication; a detected edit, creation, deletion, or directory replacement aborts the export. This advisory lock coordinates collector processes; unrelated programs that ignore it are not serialized.

Before publication, empty results, handled failures, and cancellation preserve the destination and discard staging. Publication flushes the buffer, sets permissions, syncs and closes the staging file, checks the destination again, and atomically renames it. A new destination uses a no-replace rename. The directory is then synced before success is reported. SIGINT/SIGTERM are deferred during this final publication step; a completed commit reports success. If directory sync fails after rename, exit status is `1` and the error explicitly says the new export was published. If cleanup is refused, the error identifies it. Uncaught signals, a system crash, or refused cleanup can leave a private `.collector-*` file; staging files are never automatically removed by a later run.

The staging pathname must still refer to the inode created by this process, with matching completed-file metadata and extent. A replaced or missing staging file aborts publication. Cleanup verifies ownership before unlinking and leaves a foreign replacement untouched, with an explicit error. Atomic replacement creates a new inode: ownership follows the creating process and directory, and only numeric mode bits are copied from an existing destination.

The end-to-end tests require Node.js 22 or later, `strace`, and the `openssl` CLI, with no npm dependencies:

```sh
make check
make test
```

`make check` runs Odin's strict style and vet checks. `make test` builds the optimized executable and runs it against loopback servers and isolated files. Tests cover the actual request contract, pagination, pacing, CSV, configuration, malformed input, resource limits, and the 15-second deadline. Lifecycle tests inject filesystem errors into identified syscalls and exercise signals, a kernel file-size limit, zero-umask permissions, changed destinations/directories, and overlapping collectors. All product data is synthetic. To test another build, set `COLLECTOR_BINARY=/absolute/path/to/collector` when running the Node test files.

The review also uses an instrumented build, which can be reproduced with:

```sh
odin build . -out:build/collector-asan -o:minimal -debug -sanitize:address -vet -strict-style
COLLECTOR_BINARY="$PWD/build/collector-asan" node --test tests/e2e.mjs tests/json-corpus.mjs
```

The implementation is one Odin package with explicit ownership boundaries:

| Module | Responsibility |
| --- | --- |
| `main.odin` | Arguments, process lifecycle, and exit status |
| `config.odin` | Bounded dotenv reads and credential validation |
| `cancel.odin` | Signal-safe cancellation and interruptible pacing |
| `http.odin` | HTTP client lifetime, authentication, deadlines, and bounded response bodies |
| `json.odin` | Strict JSON grammar, Unicode, and resource limits |
| `response.odin` | Search metadata and product-to-row conversion |
| `collector.odin` | Pagination and collection-wide ID tracking |
| `export.odin` | Directory locking, checked output, and atomic publication |

[REVIEW.md](REVIEW.md) records adversarial passes, verified findings, rejected probes, and the consecutive clean-pass count.

The live IndieHackers products page returned HTTP 403 with a browser challenge during the 2026-10-17 inspection. No live credentials are bundled, and fixture verification is not a claim that a live product export succeeded. With valid search-only credentials configured, the normal commands above use Algolia directly as the reference does.