Hotel prices: why scraping them is adversarial and often wrong

Hotel pricing looks like an ideal scraping target. It is public, it changes constantly, and the value of knowing it early is obvious. Two things make it harder than it looks, and only one of them is technical.

The sites do not want you doing it, and they are experienced at preventing it. More awkwardly, the number you extract is frequently not a real price: rates vary by session, by inclusion, by currency, by loyalty status and by what the site thinks you are, so a scraped figure often describes something nobody was actually offered.

What we'll cover

• The price is not a single number
• What the official routes give you
• The affiliate route, which people skip
• Where scraping sits here
• Building it with CodeWords
• Normalising into something comparable
• Alerting on a noisy series
• Storage, since the history is the value
• What breaks, and how you find out
• Making it survive
• Limits worth knowing about
• What to build first
• Frequently asked questions

The price is not a single number

The most important thing to understand before collecting anything.

Rates depend on the search. Dates, length of stay, occupancy and room type all change the answer, and a price collected for one combination says little about another.

Inclusions differ. Breakfast, taxes, city levies, resort fees, cancellation terms. Two sites showing different numbers for the same room are frequently both correct and describing different products.

Prices are personalised. Logged-in status, loyalty tier, device, apparent location and currency all affect what is shown. A scraper is a particular kind of anonymous visitor and sees the price offered to that visitor.

Availability is part of the price. A rate with one room left behaves differently from the same rate with twenty.

The consequence is not that the data is useless. It is that a pipeline must record the full context of every observation, because a price without its search parameters and inclusions is not comparable to anything, including its own earlier self.

What the official routes give you

Several exist and each has a shape worth knowing.

Supplier and aggregator APIs provide clean, structured, contractual data. They cost money, they have qualification requirements, and their coverage is defined by who is in the programme. For a commercial product this is the route that does not fall over.

Hotel and chain APIs exist for some larger groups and are useful when your interest is a specific portfolio rather than a market.

Affiliate programmes are the underused one, covered below.

Your own data if you are a property or a group, which is the cleanest source available and frequently forgotten in favour of scraping competitors.

The trade is constant: official routes give you reliability and narrower coverage, unofficial ones give you breadth and a maintenance burden that never ends.

The affiliate route, which people skip

For a large proportion of use cases this is the answer and it gets overlooked because it sounds commercial rather than technical.

Most large travel sites run affiliate programmes that include data access, precisely so partners can display rates. The access is legitimate, documented, and comes with permission. It usually requires an application and an intention to send traffic, which is a genuine requirement rather than a formality.

If you are building anything user-facing that shows hotel prices, this is almost certainly the right path. It is more work to start and less work forever afterwards, it removes the legal question entirely, and the data arrives structured rather than parsed out of a page that changes.

Worth exhausting before writing a scraper, and frequently not even considered.

Where scraping sits here

Accurately, because this is a domain with real history.

Travel sites' terms generally prohibit automated access, and they enforce it more actively than most sectors because scraped inventory data has direct commercial value to competitors. Expect blocking, challenges, and rate limiting designed to make collection expensive.

There is also a body of litigation in this specific area. Travel and aggregation businesses have been to court over automated collection more than once, and the outcomes turn on details such as whether terms were accepted, whether a technical restriction was circumvented, and the jurisdiction. This is not a settled "public data is fair game" situation.

The practical position: where you do collect from public pages, do not circumvent technical protections, do not create accounts to reach data behind a login, respect the rate at which a person would browse, and take only what you need. And be clear-eyed that this route is adversarial by design and will need continuous maintenance.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which properties and date combinations to check, how often, and what constitutes a change worth telling you about.
3. Authorize the connections to whichever data source you have arranged and wherever observations should land.
4. Describe the exceptions: a source returning nothing, a blocked request, a property that has disappeared, a value outside a plausible range.
5. Record the full search context with every observation, since a price without its parameters cannot be compared to anything later.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Normalising into something comparable

Most of the engineering effort sits here rather than in collection, and it is the part that determines whether the dataset means anything.

Resolve identity. The same hotel appears under different names, spellings and identifiers across sources. This is the hardest part and it is worth solving deliberately with coordinates and addresses rather than by string matching.

Normalise the product. Room type, occupancy, board basis and cancellation policy have to become comparable fields rather than free text.

Make the money comparable. One currency, one basis on taxes and fees, with the conversion rate recorded alongside so a historical figure can be reinterpreted.

Record what was included, as structured fields. A rate with breakfast and free cancellation is not the same product as a non-refundable room-only rate, and comparing them produces confident nonsense.

A dataset without this is a collection of numbers that cannot be compared, which is worse than a smaller dataset that can.

Alerting on a noisy series

Prices move constantly, so a threshold on absolute change produces an alert stream nobody reads.

Compare like with like, meaning the same property, same dates, same product.

Use relative movement against that series' own volatility, so a property that swings ten per cent daily does not alert on a ten per cent move while a stable one does.

Suppress round trips. A price that drops and recovers within an hour is noise.

Alert on the thing you would act on. Usually a drop below a figure you nominated, or a sustained move rather than a spike.

Rate limit yourself. One digest beats forty notifications, and an alert channel people mute has failed.

Storage, since the history is the value

A single price is close to worthless and a series is valuable, which shapes the design.

Store every observation with its timestamp and full context, append-only, rather than maintaining a current-price table. The current price is a query against the series.

Keep the raw response for a period, because parsing bugs are discovered later and a stored raw response can be reparsed while a lost one cannot be recollected. And record failures as observations too, since knowing a source returned nothing on a given day is itself information and a gap that looks like stability is misleading.

The series cannot be obtained retrospectively, which is the argument for starting collection before you have decided exactly what you will do with it.

What breaks, and how you find out

Silent emptiness. The request succeeds and returns no prices. This is the failure that goes unnoticed for weeks, and the defence is alerting on a volume drop rather than only on errors.

Layout changes, which break parsing without any error.

Blocking, which frequently returns a valid page with no data rather than an error status.

Currency or locale drift, where a source starts returning a different currency and the numbers look plausible.

Implausible values. Validate ranges; a nightly rate of zero or of six figures is a parse failure, not a bargain.

Making it survive

Exhaust the official and affiliate routes first.

Record full context with every observation.

Append, never overwrite, and keep raw responses for a period.

Alert on volume drops, not only on errors.

Validate value ranges before storing.

Limits worth knowing about

A scraped price is often not a real offer, being session and context dependent.

Travel sites actively prevent collection, and this sector has litigation history.

Identity resolution across sources is genuinely hard and determines whether the data is comparable.

Historical series cannot be backfilled.

Maintenance is permanent on any unofficial source, which is the cost people underestimate.

What to build first

A single property, a single date combination, collected daily with full context, stored append-only. Not a market, not a comparison engine. You will learn within a fortnight how much the number moves, how often the source fails, and whether the figure you are capturing is actually the one you meant.

Two habits make the difference. Record the complete search context with every observation, because a price without its dates, occupancy and inclusions cannot be compared even to its own history. And apply for affiliate or API access before writing a scraper, since it is more work once and less work forever, and it removes the question of whether you should be doing this at all.

Frequently asked questions

Why do scraped hotel prices not match what users see?

Because rates are personalised and context-dependent. Logged-in status, loyalty tier, device, location and currency all affect the price shown, and a scraper is one particular anonymous visitor.

What makes two prices for the same room differ?

Usually inclusions. Breakfast, taxes, city levies, resort fees and cancellation terms make them different products, so both numbers can be correct.

What are the official routes to this data?

Supplier and aggregator APIs, chain APIs for specific portfolios, affiliate programmes that include data access, and your own data if you operate properties.

Why use an affiliate programme?

It provides legitimate, documented, structured access with permission, which removes the legal question and the parsing burden. It requires an application and an intention to send traffic, and is frequently the right answer for anything user-facing.

Is scraping travel sites allowed?

Their terms generally prohibit it and they enforce actively, since the data has direct commercial value to competitors. This sector also has litigation history, so it is not a settled matter of public data being freely collectable.

What is the hardest engineering problem here?

Identity resolution. The same hotel appears under different names and identifiers across sources, and solving it with coordinates and addresses rather than string matching determines whether the dataset is comparable at all.

How should alerts be designed?

On relative movement against each series' own volatility, comparing identical property, dates and product, suppressing short round trips, and digested rather than sent individually.

How should observations be stored?

Append-only with a timestamp and the full search context, keeping raw responses for a period so parsing bugs can be corrected retrospectively. The current price is a query against the series.

What failure is easiest to miss?

Silent emptiness, where a request succeeds and returns nothing. Blocking often looks like this too. Alert on a drop in the volume of results rather than only on error statuses.

Can I backfill historical prices?

No. The series can only be built forward, which is the argument for starting collection before you have finalised what you will do with it.

Related reading

• Extracting creator data
• Automating a Reddit account without losing it
• Managing credentials across automations