Skip to content
◉RedLens Get RedLens on GitHub
Public Discussion Analysis

PullPush Has Newer Records—Check Your Query’s Coverage

PullPush returns October 2026 records, but that does not prove complete coverage. Diagnose missing months, query limits, and local archive gaps.

The RedLens Project

PullPush can return older Reddit data while recent months are missing because the searchable archive and the working API are separate things. A successful response proves that a request worked—not that the archive covers your requested dates.

A reported May 2025 cutoff is not a reliable current boundary. On October 9, 2026, PullPush’s newest-comment query and newest-submission query returned records dated October 4, 2026, in UTC. That establishes that newer data exists; it does not establish complete coverage of the intervening months or every subreddit.

Where the May 2025 explanation came from

A PullPush forum user reported that searchable posts ended on May 20, 2025, and comments on May 19. The user suspected maintenance, but that explanation was not an operator-confirmed diagnosis in the discussion.

Separately, commercial archive provider THINKPOL reported a July 30, 2026 check in which PullPush’s newest returned content was still from May 2025. Its warning about silently stale results describes an analytical risk, but its cutoff should not be treated as today’s permanent boundary.

Availability of newer records does not prove historical gaps were backfilled. Check the months you need rather than replacing one assumed cutoff with another.

Diagnose your missing window

Use these checks before changing providers or interpreting an empty result as no Reddit activity.

1. Check freshness without your research filters

Open the newest-comment and newest-submission queries above. They request:

  • sort=desc: newest first.
  • sort_type=created_utc: sort by creation time, not score.
  • size=1: return one record.

Read created_utc, a Unix timestamp in seconds, from the returned object. Do not substitute the time you downloaded the response. PullPush documents these parameters and the data response array in its API reference.

If an unfiltered query returns recent records but your keyword search does not, the entire archive is not simply frozen at your search’s last date. Investigate the narrower scope next.

2. Test the missing month separately

Repeat the query for each missing month using after and before, with epoch seconds rather than milliseconds. PullPush’s documentation says boundaries are inclusive: to isolate a UTC calendar month, use its first second through its last second. Start without q, author, or subreddit; then restore one filter at a time. Parameter and boundary documentation

Distinguish the outcomes:

Result What it supports
Timeout, HTTP error, invalid JSON, or a reported API error An access or service problem—not a finding about activity
Error-free response with an empty data array No matching records returned by that query
Recent records globally, none in your subreddit A scope-specific coverage or filtering question
Records at a month’s end Some coverage of that month—not proof of completeness

For a consequential study, also check a few known public post or comment IDs from the missing window using ids. Keep “not returned by PullPush” separate from “never posted.”

3. Separate API results from your local archive

If direct queries return newer records but your database does not, inspect your saved date filters, pagination, incremental cursor, and failed-request logs. PullPush documents a maximum size of 100 and no offset parameter. Its pagination instructions retain the timestamp boundary and use exclude_ids for records already collected at that boundary, avoiding skipped records that share a timestamp.

Retrying can address a failed request; it cannot create records absent from the source. Preserve the endpoint, parameters, collection time, newest returned timestamp, and errors alongside each dataset. Report missing windows as coverage limitations, not zero-activity months.

Preserve a scoped local archive

RedLens stores public Reddit records from Arctic Shift in a local SQLite file. It does not repair missing source data. Arctic Shift’s API documentation offers no uptime or performance guarantees and restricts keyword searches to supported scopes rather than global full-text search.

For an installed RedLens setup, start with a new database and topic to make the subreddit scope explicit:

redlens --db climate.sqlite track heatwave \
--query heatwave --subreddits climate \
  --sources none --days 180 --comments
redlens --db climate.sqlite show --topic heatwave --json
redlens --db climate.sqlite export --topic heatwave \
  --format jsonl -o heatwave.jsonl

These current CLI options collect matching posts in the specified subreddit and comment threads beneath them—not all comments independently mentioning the keyword. --sources none bypasses discovery, including the optional PullPush-backed global discovery route. Re-tracking an existing topic reuses its saved subreddit list.

Inspect the exported dates before plotting a trend. Keep a baseline copy before refreshing. For an account-focused workflow, see archiving a Reddit user’s history locally. The output is a documented collection of returned public records—not a claim that every Reddit discussion was captured.