08
Jul
With the rise of GenAI, the demand for data has increased dramatically, making it more valuable than ever. In the current digital era, website owners face the significant challenge of keeping their data safe from AI bots scraping their content without permission. AI companies often use content from public websites to train their large language models (LLMs). While some larger companies such as Google and OpenAI offer website operators to opt out of scraping, not all LLM developers are that transparent. This issue of web scrapping was highlighted a few months ago when Reddit struck a $60m deal with Google…