What is data scraping? Why does Elon Musk want Twitter users to limit it?

Twitter features have undergone several changes ever since Elon Musk acquired the microblogging platform last year. He has implemented many changes for users and the latest one being the daily limits on the number of tweets the users can read. As per the temporary reading limit imposed by Twitter, verified users can now read up to 10,000 posts a day while the same for unverified users is capped at 1,000, and newly registered unverified accounts can read upto 500 posts.

This announcement comes on the heels of a move where Twitter blocked access for anyone who is not logged in. Musk explained that these restrictions are a response to the aggressive Data Scraping activities of several hundred organisations, which were allegedly compromising the user experience.

What is Data Scraping?

Think of Data Scrapping as a digital scout, swiftly navigating through websites and gathering information in the same way you might take notes from a library book. This can include anything from remarks made on a blog, rates of items in an online marketplace, to shared thoughts on social platforms. The scout, in reality, is a piece of code or software that scrutinises the website's blueprint and zeroes in on the segments it's been programmed to fetch. It's a tool that allows people to accumulate large volumes of information from multiple webpages, saving the trouble of manually sifting through each page and noting the data.

As for the legal side of Data Scrapping, it's a labyrinth of regulations that hinge on aspects such as the website's terms of service, the nature of the data scraped, and the jurisdiction under which you're operating.

Data Scrapping, interestingly, can bypass APIs. An API, or Application Programming Interface, is essentially a rulebook that facilitates communication between different software applications. Earlier this year, Twitter announced a three-tier API change that began charging users for API access. The API’s pricing system was widely panned by critics and users for being exorbitant. 

While Data Scrapping can be advantageous, like procuring data for market analysis, it also comes with significant disadvantages. For instance, it can unintentionally skew the results of AI models and enable unethical practices such as identity fraud and unwanted communication.

Musk's policy gets mixed reaction

Musk's latest policy raises questions about its effectiveness. Musk himself had to repeatedly post about the latest rate constraints, adjusting them throughout the day upon realising that the lowest rate limits mean that Twitter users can browse the website for mere minutes. By limiting site access, Twitter potentially reduces ad impressions, which could deter advertisers and harm user experience. 

The situation prompted intense public scrutiny. Hashtags like "Goodbye Twitter" and “RIP Twitter” were trending on the microblogging site immediately after the rate limits were applied. With Musk having set no clear timelines about how long the rate limits will stay in place, many users may look to jump ship to Instagram's new app offering, Threads, which takes on Twitter in the microblogging space. 

Monetisation and management changes at Twitter

These rate limits are not isolated changes but are part of a series of adjustments Musk has introduced to monetise Twitter. Earlier this year, the company introduced the revamped Twitter Blue verification scheme, which charges users $8 per month.

Alongside these changes, Musk has appointed a new CEO, Linda Yaccarino. The former NBC Universal ad executive was brought on board to rebuild ties with advertisers, who had reduced their Twitter ad spend. Despite being a private company, Twitter's decision to hire Yaccarino underlines the significance of advertising revenue to the platform.

However, Musk's measures have sparked questions. Moreover, critics point out that Musk's decision to lay off over half of Twitter's staff since taking over, including key personnel, has impacted the platform's stability. Such moves have led to instances of instability, including a significant outage in March and unpaid Google Cloud bills, indicating substantial cost-cutting efforts.