Track 3 of 4
Civic Tech: turn an Irish public portal into a clean JSON feed.
Irish local government, the courts and the planning system publish enormous amounts of public information — but most of it lives in PDFs, table-fragments and clunky portals nobody can search. Your job in this track is to scrape one of them properly, structure the output, and feed it into a live network where citizens can actually find it.
What you'll learn
Civic tech isn't a different language, it's a different discipline. The scrape itself is the easy bit. The hard bits are being polite to public servers, structuring messy real-world data, handling the edge cases gracefully, and writing something that still works in six months. By the end of four weeks, you will have:
- Picked one council, court or planning source and read its HTML or API carefully enough to know what you're dealing with.
- Written a polite scraper — rate-limited, identified, robots.txt-aware, with sensible retries.
- Structured the output as JSON in a defined schema the receiving project (likely localnews.ie) can ingest without further cleaning.
- Handled the messy real-world cases: missing fields, malformed dates, duplicate listings, paginated archives.
- Documented your scraper for the next person, including how to run it, how to test it, and how to debug it when the site changes.
- Worked through at least one tricky moment when the source site changed shape mid-cohort and you had to adapt.
Cadence
Four weeks. One source, one scraper, one production handoff.
Week 1
Pick + study. Choose your source from the cohort's target list. Read its structure. Document its quirks. Write a one-page brief: what you'll scrape, what you'll output, what you'll skip.
Week 2
First scrape. A working scraper that pulls one page and outputs JSON. Doesn't have to be beautiful. Has to run. Mentor paired-coding session mid-week.
Week 3
Harden + scale. Pagination, retries, rate-limiting, error handling, schema conformance. Test fixtures so the mentor can run it locally.
Week 4
Hand off. Code review, README written, handoff to mentor for production wire-up. Reference paperwork signed.
What you'll ship in Cohort 1
If you take a Civic Tech / scraper role in Cohort 1 (Carlow on localnews.ie), your shippable is one production scraper for one Carlow public-data source. The strongest target candidates:
- Carlow County Council news — the council's news section produces ~3 items a week. Predictable structure. Good first scrape.
- Carlow County Council planning portal — daily new applications, more complex structure, higher-value output.
- Carlow Circuit / District Court list — published weekly, important to local news, currently uncovered on localnews.ie.
You'll write the scraper, output JSON in the localnews.ie schema, and hand off to Kali for the production wire-up.
Example of what shipped civic-tech work looks likeThe existing council and planning scrapers feeding localnews.ie (workers/council_news.py, workers/national_planning.py) are exactly this size and shape. Your Carlow scraper joins them when it ships.
Who this track is for
You'd suit this track if you've written at least a little Python (or JavaScript, or Ruby — we can translate) and you're comfortable with the idea of:
- Reading HTML structure carefully and noticing patterns.
- Debugging by printing things and reading the output.
- Being patient with public-sector websites that occasionally make bizarre choices.
If you've never coded, the Vibe Coding track is a friendlier entry to the same world — and Vibe Coding members frequently graduate into Civic Tech-shaped problems on later cohorts.
Reference criteria
To graduate with a public reference at builtinireland.ie/references/<you>:
- A working scraper in production (running, producing valid JSON the project ingests)
- 4 weekly check-ins attended
- Mentor sign-off after code review
- 2-paragraph self-reflection (becomes the letter text)
The reference will name the scraper, name the source, and link to the scraped output appearing on the live site. See a sample →
Cohort 1 has 1-2 scraper slots open
Civic Tech is the highest-skill role in Cohort 1 and also the most replaceable from the cohort's perspective — if you bail, we still ship everything else. Take it if you want the challenge.
Apply now →