The Good Judgment Project ( GJP ) is an organization dedicated to "harnessing the wisdom of the crowd to forecast world events". It was co-created by Philip E. Tetlock (author of Superforecasting and Expert Political Judgment ), decision scientist Barbara Mellers , and Don Moore , all professors at the University of Pennsylvania .
18-851: The project began as a participant in the Aggregative Contingent Estimation (ACE) program of the Intelligence Advanced Research Projects Activity (IARPA). It then extended its crowd wisdom to commercial activities, recruiting forecasters and aggregating the predictions of the most historically accurate among them to forecast future events . Predictions are scored using Brier scores . The top forecasters in GJP are "reportedly 30% better than intelligence officers with access to actual classified information." The Good Judgment Project began in July 2011 in collaboration with
36-646: A public forecasting tournament at the Good Judgment Open site. Like the Good Judgment Project, Good Judgment Open has questions about geopolitical and financial events, although it also has questions about US politics, entertainment, and sports. GJP has repeatedly been discussed in The Economist . GJP has also been covered in The New York Times , The Washington Post , and Co.Exist. NPR aired
54-431: A segment on The Good Judgment Project by the title "So You Think You're Smarter Than a CIA Agent", on April 2, 2014. The Financial Times published an article on the GJP on September 5, 2014. Washingtonian published an article that mentioned the GJP on January 8, 2015. The BBC and The Washington Post published articles on the GJP respectively on January 20, 21, and 29, 2015. The Almanac of Menlo Park published
72-478: A story on the GJP on January 29, 2015. An article on the GJP appeared on the portal of the Philadelphia Inquirer , Philly.com, on February 4, 2015. The book Wiser: Getting Beyond Groupthink to Make Groups Smarter has a section detailing the involvement of the GJP in the tournament run by IARPA. Psychology Today published online a short article summarizing the paper by Mellers, et al ., that wraps up
90-506: A total of about 30 team members, including the co-leaders as well as David Budescu , Lyle Ungar , Jonathan Baron , and prediction-markets entrepreneur Emile Servan-Schreiber. The advisory board included Daniel Kahneman , Robert Jervis , J. Scott Armstrong , Michael Mauboussin , Carl Spetzler and Justin Wolfers . The study employed several thousand people as volunteer forecasters. Using personality-trait tests, training methods and strategies
108-668: Is an American media company. It owns The Philadelphia Inquirer and Philadelphia Daily News . The company is owned by The Philadelphia Foundation , a nonprofit organization in Philadelphia , Pennsylvania, U.S. Philadelphia Media Network, then including the newspapers' joint web portal Philly.com , was formed and initially owned by the creditors of Philadelphia Media Holdings (PMH), acquired out of Chapter 11 bankruptcy protection . The company sold its inherited community newspaper division in December 2010. A group of local investors under
126-610: The Aggregative Contingent Estimation (ACE) Program at IARPA (IARPA-ACE). The first contest began in September 2011. GJP was one of many entrants in the IARPA-ACE tournament, which posed around 100 to 150 questions each year on geopolitical events. The GJP research team gathered a large number of talented amateurs (rather than geopolitical subject matter experts), gave them basic tutorials on forecasting best practice and overcoming cognitive biases , and created an aggregation algorithm to combine
144-525: The Economist discusses the main concepts. A Wall Street Journal article depicts it as: "The most important book on decision making since Daniel Kahneman’s Thinking, Fast and Slow ." The Harvard Business Review paired it to the book How Not to Be Wrong: The Power of Mathematical Thinking by Jordan Ellenberg . On September 30, 2015, NPR aired an episode of the Colin McEnroe Show centering on
162-492: The GJP and the book Superforecasting ; guests on the show were Tetlock, IARPA Director Jason Matheny , and superforecaster Elaine Rich. Aggregative Contingent Estimation Aggregative Contingent Estimation ( ACE ) was a program of the Office of Incisive Analysis (OIA) at the Intelligence Advanced Research Projects Activity (IARPA). The program ran from June 2010 until June 2015. The broad program announcement for ACE
180-793: The IARPA ACE program. The ACE has collaborated with partners who compete in its forecasting tournaments. Their most notable partner is The Good Judgment Project from Philip E. Tetlock et al. (winner of a 2013 ACE tournament) ACE also partnered with the ARA to create the Aggregative Contingent Estimation System (ACES). Data from ACE is fed into another program, called Forecasting Science and Technology (ForeST), which partners with SciCast from George Mason University . Philadelphia Media Network The Philadelphia Inquirer, LLC (formerly Philadelphia Media Network ( PMN ))
198-514: The company to The Philadelphia Foundation , a nonprofit organization. The Philadelphia Media Network also converted to a public benefit corporation with a charter that balances public benefit alongside shareholder returns. In 2019, Philadelphia Media Network renamed Philly.com to Inquirer.com and made the Daily News an edition of The Inquirer . Philadelphia Media Network was renamed The Philadelphia Inquirer, LLC. In January 2020, Lisa Hughes
SECTION 10
#1732872906426216-505: The corporate name of Interstate General Media LLC bought the company for $ 55 million in April 2012. Publisher and chief executive officer Greg Osberg stepped down on May 11, 2012. He was replaced by Bob Hall, 67, the publisher of the Daily News and Inquirer from 1990 to 2003, when the papers were owned by Knight Ridder . Philadelphia Media Network was purchased by Philadelphia businessman Gerry Lenfest in 2014. In 2016, Lenfest donated
234-415: The goals of ACE are "to dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts." The website claims that ACE seeks technical innovations in the following areas: There is a fair amount of research funded by grants made by
252-514: The individual predictions of the forecasters. GJP won both seasons of the contest, and were 35% to 72% more accurate than any other research team. Starting with the summer of 2013, GJP were the only research team IARPA-ACE was still funding, and GJP participants had access to the Integrated Conflict Early Warning System . The co-leaders of the GJP include Philip Tetlock , Barbara Mellers and Don Moore . The website lists
270-432: The main findings of the GJP. The project spawned a 2015 book by Tetlock and coauthored by Dan Gardner, Superforecasting - The Art and Science of Prediction , that divulges the main findings of the research conducted with the data from the GJP. Co-author Gardner had already published a book in 2010, that quoted previous research by Tetlock that seeded the GJP effort. A book review in the September 26, 2015, print edition of
288-631: The people involved with it. The results show that harnessing a blend of statistics, psychology, training and various levels of interaction between individual forecasters, consistently produced the best forecast for several years in a row. A commercial spin-off of the Good Judgment Project started to operate on the web in July 2015 under the name Good Judgment Inc. Their services include forecasts on questions of general interest, custom forecasts, and training in Good Judgment's forecasting techniques. Starting in September 2015, Good Judgment Inc has been running
306-425: The researchers at GJP were able to select forecasting participants with less cognitive bias than the average person; as the forecasting contest continued the researchers were able to further down select these individuals in groups of so-called superforecasters . The last season of the GJP enlisted a total of 260 superforecasters. A significant amount of research has been conducted based on the Good Judgment Project by
324-489: Was published on June 30, 2010. ACE funded the Aggregative Contingent Estimation System (ACES) website and interface on July 15, 2011. They funded The Good Judgment Project some time around July 2011. ACE has been covered in The Washington Post '' and Wired Magazine . The program was concluded by late 2015. The program manager was future IARPA director Jason Gaverick Matheny . The official website says that
#425574