Earlier client work · not an OmniTechs case
The register project stored a company. It does not explain how it got past a block.
A client wanted the businesses of one country as structured records. The 2022 code stores the company, the people, the activity, and progress by page. How requests were spread so a source would not block them is not on this page.
What is shown, and what is withheld
business_fetcher is a public Django repository from 3 January 2022, with 94 commits from one account. The Business model has a unique URL, a registry code, legal form, VAT fields, an employee count, and a flag for whether the record is complete. Activity stores EMTAK and NACE codes. Page and URL track which page is finished and which URL failed. There is also a Proxy model.
EMTAK belongs to Estonia, and the deploy helper names the container rikfetcherapp. This is earlier client work from before OmniTechs, not a studio customer case, and not a guide for getting around a source. The demo password and the server address in the README are not repeated here.
The record
What remained per company
The columns come from fetcher/models.py. This is the shape of the dataset, not an export.
| Part | Fields in the code | What this page leaves out |
|---|---|---|
| Business | Name, registry code, legal form, status, VAT number, staff count, email, unique URL, complete or still in process | No export and no bypass |
| Person | Name, role, and a person identifier, tied to the company | No personal data from a live register |
| Activity | Area, EMTAK code, NACE code, and a source note | No claim that every record is complete |
| Progress | A page number with a finished flag, and a URL with a failed flag | No success rate |
The job
One country, as rows instead of as websites
The founder describes a client who wanted every business in one country. The code turns that into a queue that can resume: a page is finished or not, a URL has failed or not, a company is complete or still marked in_process.
That is the useful part for a later reader. A dataset is only a dataset when you can see what is missing.
What this page refuses
A proxy table is not a method this page will teach
The founder says collection happened in several rounds so the source would not block it. The code has a Proxy table with a last-used time. This page names that fact and stops.
OmniTechs does not publish how to spread, disguise, or retry requests in order to slip past a block. When a source has an allowed API or an open file, that is the connection to investigate first. A register that refuses bulk traffic is a limit of the source, not a puzzle to win.
Start from a source that allows the read.
Describe which register or API already offers the data, and which fields you actually need.