Diário SG
São Gonçalo's official gazettes, made searchable
- Solo, end to end
- ~162,000 acts indexed
- MCP server with 12 tools
São Gonçalo publishes two official gazettes, one from City Hall and one from the City Council, as long PDFs that hardly anyone reads. I built a pipeline that downloads each edition, runs OCR on scanned pages, splits it into acts like appointments, contracts, tenders and decrees, and indexes them for Portuguese full-text search. The index holds 5,286 editions since 2010, about 162,000 acts. A second collector pulls nearly 50,000 City Council bills from 2014 on, works out each one's stage from its procedural history and links a bill to the law it became, so you can see which ones have sat in committee for months.
You can search by name, CNPJ or subject and filter by act type, gazette, agency and theme, like the environment. A company's page puts the acts that cite it next to its federal tax registry record, CGU sanctions, TCE-RJ payments, City Hall spending commitments and PNCP contracts. Dashboards rank the largest suppliers and flag patterns worth a second look, like a no-bid purchase split into smaller ones or a newly opened company that gets an environmental licence. Save a term, CNPJ or process number, or just a set of filters, and you get an email when a new edition mentions it, or follow any search over RSS.
There's also an MCP server. Each person generates a key and plugs the gazette into Claude, ChatGPT or Cursor, with 12 tools to search, read and cross-check acts. It's written in Go with clean architecture and events between the scraper and the worker, and Postgres handles search with unaccent and trigram indexes. Caching the OCR output took reindexing a month from almost 10 minutes to 5 seconds. It runs live at diariosg.com.br on free tiers: collection and data loads on scheduled GitHub Actions, the API and site on Render, Postgres on Neon. To fit the 1 GB database limit I swapped a 224 MB trigram index for a 20 MB one over the numbers in each act, with the same search results. Terraform keeps a Google Cloud version ready.
Go · React · Postgres · MCP · GitHub Actions · Render
