A Magazine by the Society of Professional Journalists


#data scraping


March 8th, 2021 • Featured, Quill Blog, Toolbox, Quill Archives, Digital Media Toolbox
SPJ Journalist’s Toolbox Tool of the Month: Scrape data from a web page/format it for editing

Government websites love to bury data in tables on web pages. Why? It satisfies legal requirements for making document public under sunshine laws, but it renders the data useless. You can’t sort or filter the data to look for trends, do math calculations to find rates and averages, and other things journalists need to find stories.