Implications of the SEC index url change

On October 5th, sec.gov blocked access to SEC filing index urls, using just accession number.

This used to work: https://www.sec.gov/Archives/edgar/data/000183988226050387/0001839882-26-050387-index.html

Now, you need cik: https://www.sec.gov/Archives/edgar/data/1666268/000183988226050387/0001839882-26-050387-index.html

The ability to access index url without a cik was almost surely a bug. In documentation, the SEC mentions requiring both accession and cik. This also entered model weights, such as Claude and GPT.

Result

Before this change, using 10 requests per second, you could detect ~40% of SEC filings seconds to minutes before they were published on the RSS feed. With Fastest SEC Filings Websocket I got ~30s faster on average for about half of filings. Metrics are on Datamule's statistics page.

This worked, as you only needed to compute the next accession number, which was 'filer cik' + 'year 2d' + 'generally a sum of filings by that filer cik for that year'.

Since ~10 filers file ~40% of filings, this made the problem require monitoring few combinations.

Now you need cik. But not of the filer. Filer cik is the company filing for e.g. Morgan Stanley. Which is not Morgan Stanley, but a specific company that sells filing services. You can still predict filings early, but now need to monitor by company, rather than by filer. This requires monitoring many combinations.

Implications

After I open sourced in January 2025, a couple trading firms reached out to me. They were unaware of the accession only approach, and so were using cik + accession, bypassing SEC ratelimits with many thousands of proxy ips.

I estimate that a naive method, to achieve ~70% of filings detected early, requires ~10k IPs. Going off a naive estimate of $0.50/IP, yields a sum of ~$5k/month. Affordable for trading firms. Not for retail traders.

Future of affordable url prediction

I think I can get substantial coverage, respecting the SEC's rate limit, via machine learning.