Scraping tutorial

Requests based scraping tutorial

You want to start scraping? Well this guide will teach you, and not some baby selenium scraping. This guide only uses raw requests and has examples in both python and kotlin. Only basic programming knowlege in one of those languages is required to follow along in the guide.

If you find any aspect of this guide confusing please open an issue about it and I will try to improve things.

If you do not know programming at all then this guide will not help you, learn programming! first Real scraping cannot be done by copy pasting with a vauge understanding.

Once you’ve read and understood the concepts behind scraping take a look at a provider for CloudStream. I added tons of comments to make every aspect of writing CloudStream providers clear. Even if you’re not planning on contributing to Cloudstream looking at the code may help.

Edit on GitHub

If you see a mistake here, you can open a pull request
Children

Starting

Scraping is just downloading a webpage and getting the wanted information from it. As a start you can scrape the README.md I’ll use khttp…

Using APIs

About Whilst scraping a site is always a nice option, using it’s API is way better. And sometimes its the only way . Anyways, this guide…

Devtools detector

TL;DR: You are going to get fucked by sites detecting your devtools, the easiest bypass for this is using a web sniffer extension Many sites…

Disguishing your scrapers

Disguishing your scrapers Why is scraping not appreciated? It obliterates ads and hence, blocks the site revenue. It is more than usually…

Finding video links

Finding video links Now you know the basics, enough to scrape most stuff from most sites, but not streaming sites. Because of the high costs…