Business Challenge
Targeted News Service (TNS) is a US-based specialized news wire service. It delivers open source intelligence in the form of federal and congressional press releases, US House bills, Senate bills, federal contract awards, federal grant opportunities, and patents. TNS clients include major newspapers and other information products. They rely on TNS for timely and consistent delivery on these niche but highly valuable information streams. However, tracking these developments isn’t easy. TNS uses Contify to aggregate open source intelligence from government and regulatory websites, press releases from company websites and patent filings from the USPTO website. Once aggregated, this information is tagged with relevant metadata and formatted using a combination of technology and human editing.
The Solution
We’ve built a customized News API solution that enables TNS to deliver open-source intelligence from over 10,000 web links on a daily basis. Here’s how it works:
- Content aggregation: We have developed custom scrapers for different websites to source information. Government and federal agency are extensive websites with several sections, but TNS needs information from only specified sections. Our custom scrapers monitor only specific pages and leave out the rest. We also have a system in place that flags any new information posted on the pages being monitored. In addition, whenever there is any change in the layout of the page, the scrapers are immediately updated to ensure continuous aggregation of content on a real-time basis.
- Contextual sourcing: TNS has well-established guidelines for what qualifies as open-source intelligence and what doesn’t. For instance, contract solicitations and introductions of house bills qualify but stories about hunting or announcements targeting internal audiences are not intelligence. Here’s an example of how our system handles this. A solicitation by the U.S. Department of Veterans Affairs for engineering services was allowed. Similarly, a resolution introduced by Rep. Wilson on the Temporary Protected Status for Haitian Nationals was allowed. However, a news release issued by the Montana Fish Wildlife and Parks asking the public to help identify those responsible for the illegal killing an elk was rejected by our platform.
- Editorial support: Once the information has been aggregated, it needs to be formatted to meet TNS editorial guidelines. For example, statements issued by agencies or government officials need to carry a headline and summary that clearly explains the subject on which the statement was issued and by whom. Consider a story which carried this headline – ‘Women’s Caucus Leadership Condemns Sexual Assault in the Capitol Community’. The headline doesn’t specify who condemned the sexual assault and what is their title. We have developed a custom module which identifies the person making the statement in the story, their title and then generates:
- A headline: Legislative Women’s Caucus Issues Statement Condemning Sexual Assault in Capitol Community
- A summary: The California Legislative Women’s Caucus issued the following statement by Assembly woman Cristina Garcia, D-Bell Gardens, Chair of the Legislative Women’s Caucus and Senator Connie M. Leyva, D-Chino, Vice-Chair of the Legislative Women’s Caucus.
When our system is unable to identify the person in the article, it is flagged for review by an editor.
- Custom processing of patent filings: TNS tracks patents being filed by companies in different states in the US. Once we source these patents, our text mining algorithms identify the state in which the patent was issued. Using this process, we are able to organize patents by state. Another challenge is the overtly scientific language of the patent filings. To understand what the patent is about, each document link needs to be opened. We have addressed this, by developing a module to generate logical headlines and summaries.
What Made our Solution Stand Out
Contify News APIs have helped Targeted News Service build a reliable open-source intelligence solution. Our deep technical capabilities enable TNS to confidently scale its operations while our analyst support services ensure that they are able to sustain quality without adding operational costs.
