Writing a web crawler with Nim

Here’s how you can easily build a web crawler with Nim. To see a full real example, you can review how I wrote Torrentinim, leave a star if you learn something from it! https://github.com/sergiotapia/torrentinim Let’s create a simple type called Torrent to keep the data we crawl in a nice object. import times type Torrent* = object name*: string source*: string uploaded_at*: DateTime canonical_url*: string magnet_url*: string size*: string seeders*: int leechers*: int proc newTorrent*(): Torrent = result = Torrent(uploaded_at: now()) We need to write a crawler, let’s build one that grabs the latest torrent uploads on eztv.

How to parse RSS feeds with Nim

You have some RSS and you need to parse it into a Nim object. For this example, we’re going to be using the following RSS feed: <?xml version="1.0" encoding="UTF-8"?> <rss xmlns:torrent="http://xmlns.ezrss.it/0.1/" version="2.0"> <channel> <title>TV Torrents RSS feed - EZTV</title> <link>https://eztv.io/</link> <description>TV Torrents RSS feed - EZTV</description> <lastBuildDate>Sun, 22 Dec 2019 12:37:01 -0500</lastBuildDate> <item> <title>Five Day Biz Fix S01E03 480p x264-mSD</title> <category>TV</category> <link>https://eztv.io/ep/1400104/five-day-biz-fix-s01e03-480p-x264-msd/</link> <guid>https://eztv.io/ep/1400104/five-day-biz-fix-s01e03-480p-x264-msd/</guid> <pubDate>Sun, 22 Dec 2019 11:22:52 -0500</pubDate> <torrent:contentLength>266476297</torrent:contentLength> <torrent:infoHash>92424828AFFE4C897D5F4171BCE59B743CAAFA6C</torrent:infoHash> <torrent:magnetURI><!

Genserver Webcrawling in Pure Elixir

A lot of people think that you need a full Phoenix application to create robust background processing applications. I guess coming from Rails where most people just reach for Rails by default, it’s not that strange. Think about the last time you’ve written Ruby code without using Rails. Strange right? Well in Elixir land, developers are trying to shake off the feeling that Phoenix is Elixir and Elixir is Phoenix. Phoenix is beautiful, but Elixir is also drop-dead gorgeous.