June 20, 2022

Why web data could be the key to startup decision-making

We all know that data is power. But what about web data?


5 min read

In partnership with

Bright Data

Scroll. Click. Add to basket. The internet is a huge part of our everyday lives, so it’s vital startups understand how customers interact with their businesses online. One way to do this is through public web data — data that is sourced from websites and can be used as an indicator for how users interact with certain products or offerings.

This could be anything from price points, consumer reviews and reactions, to what marketing channels customers use the most, what choices they’re making and how they feel about them. Collecting these insights enables startups to understand more about their market landscape and their target customer journey, which helps them to make changes that improve their experience and inform their future business strategy.

“Any business of any size wants to make sure they fully understand their market landscape before making any decision or deploying an operational system,” Or Lenchner, CEO at web data collection company Bright Data, tells Sifted. “Collecting web data in real time allows companies to do just that. It allows them to know with certainty rather than guess what their next move should be.”


Put simply, if business-focused decisions must be based on precise, reliable information, web data is your know-all strategic adviser to win more market share. So what does your startup need to know about web data? 

Don’t just collect, analyse too

If you’re a startup — no matter your current stage — web data can be your eyes and ears. But, if you’re collecting public web data, you’ll only reap any value if you’re receiving it in a structured format and analysing it too. 

To do this, Amy Sharif, head of data science operations at decision intelligence startup Peak, suggests looking into three key areas: whether your website is driving the right customer behaviour, if you’re bringing your target customers to the website and whether you are showing the right products and offerings.

“Web data is incredibly rich, meaning there are endless actionable insights you can gain from it,” says Sharif. “What people say they do versus what they actually do can be wildly different. This is why analysing data straight from the source — like web data — can be more reliable than other methods, such as market research.”

Actionable insights could include which product is getting the least interest online, what proportion of customers are new to your website versus repeat visitors or what the last page people tend to visit is before leaving your site.

Sifted Newsletters

Startup Life

Every Wednesday

How (and how not) to run a startup.

What people say they do versus what they actually do can be wildly different. This is why analysing data straight from the source — like web data — can be more reliable than other methods

Real world web data applications

One startup using web data is New York-based startup Mathison, which works with Bright Data and teams up with brands to help improve their recruitment methods. To search for suitable candidates, Mathison uses web data to reach people who are sometimes hidden from the initial candidate pool and research parameters. 

This public web data can be found via LinkedIn, Twitter or publicly available forums. The initial search that most companies do is often low level, whereas Bright Data goes broader to access multiple open sites. Mathison then takes this data to build a profile of a candidate, build a list of all suitable potential recruits and then set up alerts when that same type of candidate is available for a role.

“This process is automated and uses web data collection tools, making it much simpler and quicker to address those immediate recruitment needs,” says Lenchner. “What once took days, if not weeks, is now reduced to mere minutes of a quality-driven process, producing reliable and structured data that is easy to work with.”

Other startups tap into web data to research new innovations that directly relate to their market, including how competitor’s goods and services are being priced, as well as their campaigns or offers.

Finessing web data collection

As a startup, you need to be flexible around changing customer demands. Collecting and analysing data regularly can help you to make decisions at pace — and once you’ve created a useful piece of analysis, it will become much faster to repeat in the future.


But how can startups begin their web data collection operation?

“My main piece of advice would be to start with high level data to inform larger, strategic decisions and move to more granular data over time,” says Sharif. 

“Data is extremely useful, but can also be overwhelming if you try to answer too many questions all at once. It’s important to weigh up the ease of collecting and analysing certain data versus the impact any insights would have. This will help you to prioritise where to start.”

Sharif adds that she would always recommend a quick "health check" of your website first. This can help highlight issues you weren’t aware of, such as a high bounce rate or low conversion rate. 

After this, it’s good practice to decide what outcomes you want to capture data about, says Sharif. 

Data is extremely useful, but can also be overwhelming if you try to answer too many questions all at once

For example, if you’re focusing on growing your brand awareness, Sharif recommends focusing on outcomes associated with the early stages of the customer journey, such as number of web visitors and bounce rate. If your main goal is to grow profit, you may be more interested in data closer to the end of the funnel, like conversion rates.

“Any company of any size should start by defining the data that will make a difference to their business efforts,” says Lenchner. “The data is there for the taking; all you need to do is ask the right questions for your business and try to reach that data manually on a small scale. Then, you can go bigger and automate additional processes.”