When you host your comments on a third-party platform, you're renting your community. When the platform changes its terms, adds tracking, degrades performance, or simply shuts down — your community goes with it.
For the All About archive, we had nearly 200,000 comments across two sites, all hosted on Disqus. The comments were part of the editorial record — readers correcting details, sharing experiences, debating Nokia's strategy decisions in real-time. Losing them would be like archiving a newspaper but throwing away the letters page.
Why Remark42
Remark42 is self-hosted, open-source, stores everything in a local BoltDB database, supports OAuth login (Google, GitHub, Microsoft), and has a tiny footprint. No external dependencies, no tracking, no ads. Perfect for an archive.
The critical technical decision was same-origin proxying. Remark42 creates an iframe to render comments. If the comments server is on a different domain, Safari's Intelligent Tracking Prevention blocks third-party cookies, breaking authentication. The solution: serve Remark42 at allaboutsymbian.com/comments via Caddy reverse proxy, making it same-origin with the main site.
The AAS migration
All About Symbian had 102,496 comments across 3,570 threads. The migration had three phases:
Phase 1: Embed swap. Replace the Disqus <div id="disqus_thread"> and lazy-loading script with the Remark42 equivalent on every article page. 13,500 spidered HTML files, each modified in place.
Phase 2: Import. Disqus provides an XML export. But the URLs in the export don't match the archive URLs — they reference the old CMS paths, use http:// instead of https://, and include articles that no longer exist. A rewriter script maps each thread to its current URL using the CMS database: article ID → furl → current section → canonical URL.
Phase 3: Spam filtering. Disqus's export includes spam it had filtered at display time but kept in the database. We removed 8,000 Disqus-flagged spam comments plus 674 additional spam users identified by heuristic patterns (CJK character spam, link farms, known spam keywords).
The AAWP migration
All About Windows Phone was the bigger challenge. 95,338 comments, a separate Remark42 instance (different domain needs its own OAuth callbacks), and a 163MB XML export file.
The import hit two problems:
Problem 1: Remark42 crashes on 404 pages. During import, Remark42 fetches each page to get its title. When the page returns 404, the response handler dereferences a nil pointer and panics. The entire import crashes partway through. The fix: strip all threads with unresolvable URLs from the XML before importing — reducing 249,000 threads to 9,217 threads that actually map to existing archive pages.
Problem 2: The file is too big. At 163MB (reduced to 64MB after stripping), the XML exceeds internal limits when uploaded via the API. The solution: bypass the CLI tool entirely and POST directly to the container's internal API with a longer timeout.
The import took about 10 minutes to process, working through 9,217 threads and writing 89,483 comments to the BoltDB database.
The listing page updates
Comments aren't just for article pages. The listing pages — where readers browse through articles — show comment counts next to each headline. On AAS, the original spider captured these from Disqus. On AAWP, they all said "Comments" with no count.
A batch query script hits Remark42's /counts API with groups of 50 URLs, retrieves the comment count for each, and rewrites the listing page HTML. Rate limiting at 1 request per second keeps the API happy. 2,330 articles now show real counts across 623 listing pages.
Historical comments and GDPR
The imported comments are maintained under the UK GDPR archiving exemption (Article 89), which provides specific provisions for archiving in the public interest. Users can request deletion of their comments via the site contact page.
What we preserved
Between the two sites:
- 191,979 comments imported and self-hosted
- 6,000+ discussion threads preserved
- Zero dependency on external services
- Full user authentication via OAuth (no anonymous commenting)
- Daily automated backups of the BoltDB database
The comments are no longer rented. They're part of the archive, served from our own infrastructure, backed up to our own storage. The community discussion from 2011 to 2022 is preserved alongside the editorial content it responded to.
New writing by email
Occasional pieces on product, technology and AI — and how they actually play out in practice.
