A Magazine by the Society of Professional Journalists


#data scraping


July 24th, 2023 • Quill Archives, Digital Media Toolbox
Making use of open source

Many major news stories begin their journey to public consciousness via social media. Witness the cellphone video shot by a bystander showing the killing of George Floyd and the videos of the Capitol insurgency of Jan. 6, 2021. With its vast and growing palette of digital tools, such open source intelligence has become a forensic art, applying to both journalism and criminal investigations. 


March 8th, 2021 • Featured, Quill Blog, Toolbox, Quill Archives, Digital Media Toolbox
SPJ Journalist’s Toolbox Tool of the Month: Scrape data from a web page/format it for editing

Government websites love to bury data in tables on web pages. Why? It satisfies legal requirements for making document public under sunshine laws, but it renders the data useless. You can’t sort or filter the data to look for trends, do math calculations to find rates and averages, and other things journalists need to find stories.