The following are some of the best government-related sites for finding data and conducting research.
The following overview is excerpted from the Legal Guide of the Digital Media Law Project, run through Harvard’s Berkman Center. The project offers a wide variety of advice on the legal aspects of reporting.
_____
Data can make a story. It can be the backbone of an investigation, and it can lead to new insights and new ways of thinking. Unfortunately, the data you want isn’t always readily available. It’s often on the web, but it isn’t always packaged up and available for download. In cases like these, you might want to leverage a technique called web scraping to programmatically gather the data for you.
Algorithms — a series of steps undertaken to solve a particular problem — are an integral element of our digital environment. Operating in search engines, personalized news systems, global financial markets, political campaigns and many more areas, algorithms hold ever-increasing power in society, as they steer decisions and make choices about what is or isn’t important. However, as algorithms become even more complex, concerns continue to grow over their lack of transparency.
The push for open government and open data by federal officials, as well as authorities across many states and cities, can seem an unmitigated good. Talk to journalists, however, and there are myriad areas where they believe government at all levels is still being less than transparent — and less than helpful in revealing facts that the public is entitled to know.
Journalists constantly face the challenge of explaining why things happened: What were the factors in an election victory? What are the reasons behind housing segregation in a city? What is the explanation for a low-performing school? In daily journalism, we are often content to quote relevant sources or officials, and let them do the “explaining.”
Journalists are often thought of as being “word people,” and however true that may or may not be, even the most diligent reporter can blanche when faced with a thicket of figures. But sidestepping or downplaying numbers can be perilous.
Appropriately and accurately used, they can be the very foundation of a story — a project is over or under budget; students’ graduation rates are above or below average; prices are collapsing or spiking. It can even tell you when something is fact or folly.
