Re: [External] warc files

From: akierig <akierig_at_nyob>
Date: Thu, 18 Jun 2026 19:01:48 -0500
To: CODE4LIB_at_LISTS.CLIR.ORG
On 2026-06-18 (KW 25) at 11:11:18 (-0500) Eric Lease Morgan wrote:

> Thank you for the prompt replies, but I still don't see any URLs 
> pointing to warc files, sans warc files terabytes or petabyte in size. 
> There are many descriptions of sites that have been converted into 
> warc files, but no access to the warc files themselves. I'd like to 
> play with a dozen or so warc files, not of my own design, that are in 
> the few gigabyte size range. What am I missing? --E "Still 
> Embarrassed" M

it's not multi GB but here are two I made a while back if you want them.

https://nextc.anelki.net/s/tDQ8arEmEceoMFo

probably made with something like...

```
wget -pkrm --warc-cdx --warc-file=wallyblog -e robots=off 
https://inodeblog.com
```

it's not that hard to just make one with that command. just pick a site 
(and preferably use a residential internet connection).

ander kierig
Received on Thu Jun 18 2026 - 20:00:28 EDT