Improving Education
Accountability and Evaluation
Lessons from Around the World

October 3-5, 2012
Center for American Studies
Via Michelangelo Caetani 32
00186 Rome, Italy

Tuesday, October 2, 2012
17:00-19:00 Registration

Reception with Light Hors-d'Oeuvres

At: Italian Ministry of Education, University and Research,
Viale Trastevere 76/a, 00196 Roma
access limited to the participants registered online

Wednesday, October 3, 2012
8:30-9:00 Registration
9:00-9:30 Welcoming Speakers
- Room A -

Marco Rossi Doria � Under-Secretary of State, Ministry of Education, University and Research
Annamaria Leuzzi � PON Management Authority, Ministry of Education, University and Research
Douglas Besharov � University of Maryland; Association for Public Policy Analysis and Management
9:30-11:00 Plenary - Session 1: "School Accountability through Performance Monitoring"
Chaired by Jane Hannaway, American Institutes of Research: Association for Education Finance and Policy
- Room A -

School Accountability and Monitoring Systems: Insights from the U.S. and Other Countries
Helen Ladd, Duke University; Association for Public Policy Analysis and Management

Accountability in Education Around the World: Lessons from International Achievement Tests
Ludger Woessmann, University of Munich
Abstract | Presentation

11:00-11:15 Coffee Break
11:15-13:15 Parallel Sessions

Session 1.1: Building and Interpreting Scientific Evidence

Chair: Kentaro Yamamoto � Educational Testing Service
- Room A -

Review on Evaluation and Assessment Frameworks for Improving School Outcomes
Paulo Santiago � OECD

Educational Evaluation and Assessment in the Netherlands
Jaap Scheerens, Melanie Ehren, Peter Sleegers and Renske de Leeuw � University of Twente

The Heterogeneity of "Private School Effect" in Italy
Tommaso Agasisti, Samuele Murtinu and Piergiacomo Sibiano � Politecnico di Milano

Experimental Evidence on the Effect of Childhood Investments on Postsecondary Attainment and Degree Completion
Susan Dynarski and Joshua Hyman � University of Michigan; Diane Whitmore Schanzenbach � Northwestern University

Is it Live or is it Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning
David Figlio � Northwestern University; Mark Rush and Lu Yin � University of Florida

The Effects of Student Coaching in College: An Evaluation of a Randomized Experiment in Student Mentoring
Eric Bettinger and Rachel Baker � Stanford University

Session 1.2: Accountability: School Autonomy and Management
Chair: Helen Ladd - Duke University; Association for Public Policy Analysis and Management
- Room B -

Organizational Turnaround and Educational Performance: The Impact of Performance-Based Monitoring Analysis Systems (PBMAS)
Amanda Rutherford � Texas A&M University

Development of Performance Measures for a Small After-school Program Serving South Asian Youth in New York City, a Case Study
Swati Desai � State University of New York at Albany

Examination of School-Based Management in Indonesia
Rita Karam, Georges Vernez and Jeffrey Marshall - RAND

School Autonomy and Student Achievement: An International Study with a Focus on Italy
Angelo Paletta � Universit� di Bologna; Maria Magdalena Isac � Joint Research Centre, CRELL
Paper | Presentation

Planning and Working On School Improvement. Preliminary Evidence from the Quality and Merit Project in Italy
Andrea Caputo � INVALSI; Sara Mori � INDIRE; Valentina Rastelli � INVALSI

A Performance-Based Evaluation Model for Rewarding Merit in Italian Schools
Donatella Poliandri, Paola Muzzioli, Isabella Quadrelli, and Sara Romiti � INVALSI

Session 1.3: Teachers: Measuring and Ensuring Quality
Chair: Alberto Martini - Universit� del Piemonte Orientale
- Room C -

Investigating the Role of Human Resources in School Turnaround: Evidence from Two States
Michael Hansen � American Institutes for Research

Teacher Training, Extra Education and Students' Achievement: The Evaluation of a Program for Promoting Students' Performances in Italian Lower Secondary Schools
Elena Claudia Meroni � Padua University; Giovanni Abbiati � University of Milan

Under Pressure: Job Security, Resource Allocation, and Productivity in Schools Under No Child Left Behind
Randall Reback � Columbia University; Jonah Rockoff � Columbia University and NBER; Heather Schwartz � RAND Corporation

A Composite Estimator of Effective Teaching
Kata Mihaly, Daniel McCarrey, Douglas O. Staiger, and J. R. Lockwood � RAND

Teacher Assessment and Students' Performance in OECD-PISA 2009
Brunella Fiore, Isabella Romeo � University of Milano-Bicocca

Evaluating Mathematics Knowledge for Teaching as an Outcome of Pre-Service Teacher Education: Lessons from the High Performing Countries in the Teds-M Study
Maria Teresa Tatto � Michigan State University; Michael Rodriguez � University of Minnesota;

13:15-14:30 Lunch on Premises
14:30-16:00 Plenary - Session 2: "Improving Teacher and Administrator Quality"
Chaired by Bert Creemers - University of Groningen
- Room A -

Educator Effectiveness: Measures, Policies and Debates
Jane Hannaway - American Institutes of Research; Association for Education Finance and Policy

Improving Teaching Quality: Promoting a Dynamic Approach to Teacher Professional Development
Leonidas Kyriakides - University of Cyprus

16:00-16:15 Coffee Break
16:15-18:30 Parallel Sessions

Session 2.1: Accountability: Unintended Consequences
Chair: David Figlio - Northwestern University
- Room A -

When Accountability Strategies Collide: Do Policy Changes that Raise Accountability Standards Also Erode Public Satisfaction?
Rebecca Jacobsen, Andrew Saultz and Jeffrey W. Snyder � Michigan State University

School Accountability: How Can We Reward Schools and Avoid Pupil Selection?
Erwin Ooghe � KU Leuven and IZA; Erik Schokkaert � KU Leuven

When the Cat Is Near, the Mice Won�t Play: The Effect of External Examiners in Italian Schools
Marco Bertoni � University of Padova; CEP-LSE; Giorgio Brunello � University of Padova; IZA; and CESifo; Lorenzo Rocco � University of Padova

Incentives, Information, and Ideals: The Use of Economic Theory to Evaluate Educational Accountability Policies
Andrew McEachin - University of Virginia

The Failure of Educational Accountability to Work as Intended in the United States
Sharon L. Nichols � University of Texas at San Antonio

Outcome-Based and Competition-Based Policies of School Evaluation: A Comparison of School Performance and Perverse Effects in Two Mid-sized Hungarian Towns
D�niel Horn � European University Institute; Lajos B�dis � Corvinus University;

Session 2.2: Translating Research Results Into Practice
Chair: Susan Dynarski - University of Michigan
- Room B -

Exploring Politics, Accountability and Evaluation Use in the Mexican Federal Government Education Programs
Gabriela P�rez Yarahu�n � Universidad Iberoamericana

The Causal Effect of Class Size on Pupils� Performance: Evidence from Italian Primary Schools
Larysa Minzyuk and Felice Russo � University of Salento

Creating a Performance Culture: Anxiety, Climate and Performance Management Reform
Kate Destler - University of Washington

Career Pathways as a Framework for Program Design and Evaluation
David J. Fein � Abt Associates Inc.

Moving Matters: The Causal Effect of School Mobility on Student Performance
Amy Ellen Schwartz and Leanna Stiefel � New York University

Preventing and Re-integrating Early School Leavers: A Comparative Meta Evaluation of Policies Implemented in 7 European Member States.
Anja Meierkord and Massimiliano Mascherini � EUROFUND
Paper | Presentation

Session 2.3: Innovation: Changes to Policy and Curriculum
Chair: Daniele Checchi - Universit� degli Studi di Milano; IZA
- Room C -

Can Formal-Informal Collaborations Improve Science Literacy in Urban Middle Schools? The Urban Advantage Middle School Science Initiative in New York City
Meryle Weinstein, Emilyn Ruble Whitesell, and Amy Ellen Schwartz � New York University

Preliminary Evidence from the M@t.abel Teacher Professional Development Program in Italy
Gianluca Argentin � Universit� di Milano Bicocca; Aline Pennisi � Ministero dell�Economia e delle Finanze; Daniele Vidoni � INVALSI; Giovanni Abbiati � Universit� degli Studi di Milano; Andrea Caputo � INVALSI

Improving Reading Comprehension by Fostering Children�s Engagement with Books During Summer Vacation: A Cluster-Randomized Trial Comparing Strategy and Text Structure Instruction in High Poverty Elementary Schools
James S. Kim, Helen Kingston, Lisa Foster � Harvard University; Thomas G. White - University of Virginia

Standing on the Shoulders of Chess Masters: Using RTCs to Evaluate the Effects of Including Chess in the Italian Primary School Curriculum
Gianluca Argentin � Universit� di Milano Bicocca; Alberto Martini � Universit� del Piemonte Orientale; Barbara Romano � Universit� di Genova; University of Pennsylvania

Early High School Interventions to Increase Students� Access to Post-secondary Education: Experimental Impacts from Canada�s Future to Discover Project
Reuben Ford � Social Research and Demonstration Corporation

Synthesis of Findings from 15 Years of Educational Reform in Thailand: Lessons on Leading Educational Change in East Asia
Philip Hallinger � Hong Kong Institute of Education

20:00-23:00 Social Dinner (please register at registration desk)
Speech by Elizabeth King, World Bank
Thursday, October 4, 2012
9:00-10:30 Plenary - Session 3
- Room A -

"Promising Innovations Inside and Outside the Classroom"

Chaired by Neil Gilbert - University of California at Berkeley

Digital Information Literacy: Challenges and Opportunities for 21st Century Education
Jean-Fran�ois Rouet - University of Poitiers; National Center for Scientific Research

Outgrowing the Mode Effect Study of Paper and Computer Based Testing
Kentaro Yamamoto - Educational Testing Service

10:30-10:45 Coffee Break
10:45-13:00 Parallel Sessions Session 3.1: Accountability: Achievement Gap
Chair: Daniele Vidoni - INVALSI
- Room A -

Does Accountability Narrow Achievement Gaps? Evidence from the United States in the Era of No Child Left Behind
Sean Reardon, Erica Greenberg, Demetra Kalogrides, Kenneth A. Shores and Rachel A. Valentino � Stanford University

At-Risk Student Averse: Risk Management and Accountability
Julian Vasquez Heilig, Michelle Young and Amy Williams � The University of Texas at Austin

A Way to Resilience: How Can Italian Disadvantaged Students and Schools Close the Achievement Gap?
Tommaso Agasisti � Politecnico di Milano; Sergio Longobardi � University of Naples "Parthenope"

Science Competencies Across PISA OECD Countries: Comparing Exceptionally High and Low Performers
Fabio Alivernini and Sara Manganelli � INVALSI

Why Do Northern European Secondary Schools Outperform American Secondary Schools?
John H. Bishop � Cornell University and ILR

Session 3.2: Innovation: School Choice and Market Based Incentives
Chair: Jean-Fran�ois Rouet, University of Poitiers; National Center for Scientific Research
- Room B -

The Impact of High School Choice on Mediators of Student Success
Sean P. Corcoran, Lori Nathanson and James Kemple � New York University

Do KIPP Schools Boost Student Achievement?
Philip M. Gleason, Christina Clark Tuttle, Brian Gill, Ira Nichols-Barrer and Bing-ru Teh � Mathematica Policy Research

Choices for Studying Choice: Assessing Charter School Effectiveness Using Two Quasi-Experimental Methods
Devora H. Davis and Margaret E. Raymond � Stanford University

Do Charter Schools Improve Student Achievement? Evidence from a National Randomized Study
Melissa A. Clark, Philip Gleason and Christina Tuttle � Mathematica Policy Research; Marsha K. Silverberg � Institute of Education Sciences

School Choice and School Accountability: Evidence from a Private Voucher Program in Milwaukee, Wisconsin
Deven E. Carlson � University of Oklahoma; Joshua M. Cowen � University of Kentucky; David J. Fleming � Furman University

Performance Based Incentives for Learning in the Mexican Classroom
Andrew Christensen, Brian Fuller � Foundation Escalera; Victor Steenbergen � London School of Economics; Alison Hamburg � Columbia University

Session 3.2: Teachers: New Challenges
Chair: Roel Bosker University of Groningen; Groningen Institute for Educational Research
- Room C -

Render unto Primary the Things which are Primary�s: Inherited and Fresh Learning Divides in Italian Lower Secondary Education
Gianfranco De Simone � Fondazione Giovanni Agnelli

Immigrant Background Peer Effects in Italian Schools
Dalit Contini � University of Torino

Teaching Digital Skills to "Digital Natives": Does School Play a Role?
Gianluca Argentin and Marco Gui � University of Milan-Bicocca; Chiara Tamanini � IPRASE

Parental Empowerment in Mexico: Randomized Experiment of the "Apoyos a la Gestion Escolar (AGE)" in Rural Primary Schools in Mexico
Paul Gertler � University of California at Berkeley; Harry Anthony Patrinos � World Bank; Eduardo Rodr&iccute;guez-Oreggia � Tecnol&occute;gico de Monterrey, Campus State of Mexico

Issues in the Evaluation of a Program to Promote Educational Achievements of Ethiopian-Israeli High School Students
Dalia Ben-Rabi, Viacheslav Konstantinov, Ruth Baruj and Miriam Cohen-Navot � Myers-JDC Brookdal

13:00-14:15 Lunch on Premises
14:15-15:45 Session 4: "Building and Interpreting Scientific Evidence"
Chaired by Alexei Monsarrat � Atlantic Council

Towards Evidence-Based Education
Roel Bosker � University of Groningen

What Works in Education: Don't Let Perfection be the Enemy of the Good
Grover Whitehurst � The Brookings Institution

15:45-16:00 Coffee Break
16:00-18:15 Parallel Sessions Session 4.1: Accountability: Performance Measures and Evaluation
Chair: Sean P. Corcoran New York University
- Room A -

The Impact of Selection of Student Achievement Measurement Instrument on Teacher Value-Added Measures
James L. Woodworth � Hoover Institute, Stanford University; Wen-Juo Lo � University of Arkansas; Joshua B. McGee � Laura and John Arnold Foundation; Nathan C. Jensen � Northwest Evaluation Association

One-year Value-added School Effects from Various Models and their Inter-temporal Variability: Evidence from China
Pai Peng and Eckhard Klieme � German Institute for International Educational Research

Student literacy one year later. On school value added estimation using PISA-OECD
Massimiliano Bratti and Daniele Checchi � Universit� degl� Studi di Milano; IZA

Patterns of Value-Added Creation in the Transition from Primary to Lower Secondary Education in Italy
Gianfranco De Simone and Andrea Gavosto � Fondazione Giovanni Agnelli

The Infrastructure of Accountability: Examining the Governance of Longitudinal Data Systems
Stacey A. Rutledge � Florida State University; Dorothea Anagnostopoulos and Rebecca Jacobsen � Michigan State University

The Effects of Texas's Targeted Pre-kindergarten Program on Academic Performance
Rodney J. Andrews � The University of Texas at Dallas and The Texas Schools Project; Paul Jargowsky, Rutgers UniversityCamden; Kristin Kuhne � Communities Foundation of Texas

Session 4.2: Building and Interpreting Scientific Evidence
Chair: John Bishop - Cornell University
- Room B -

Reviewing Systematic Reviews: Meta-Analysis of What Works Clearinghouse Computer-Assisted Reading Interventions
Andrei Streke � Mathematica Policy Research; Tsze Chan � American Institutes for Research Paper

Reinforcing Evidence-Based Policymaking in Education: Methodological Developments at CRELL, Centre for Research on Lifelong Learning Based on Indicators and Benchmarks
Andrea Saltelli � Joint Research Centre, Unit of Applied Statistics and Econometrics

Evaluation of the Ontario Ministry of Education�s - Student Success / Learning to 18 Strategy
Ungerleider, C., Baumann, R., B&eccute;langer, J., Cartwright, F., Eaton, I., Guerriero, S. and T. Lavin � Directions-EPRG

How Salient are Performance Incentives in Education? Evidence from North Carolina
Thomas Ahn � University of Kentucky; Jacob L. Vigdor � Duke University and NBER

Evaluating the "Programa Mais Sucesso Escolar": Lessons Learned from Evaluating the Impact of a Portuguese National Educational Policy for Compulsory Education
M. Clara Barata, M. Manuela Calheiros, Joana Nunes Patr&iccute;cio, Jo�o Gra�a and M. Lu&iccute;sa Lima � ISCTE-Lisbon University Institute and CIS-IUL

Equity in School: A Challenge for Regional Based Educational Systems
Patrizia Falzetti and Roberto Ricci � INVALSI

Session 4.3: Teachers: Recruitment, Retention, and Distribution
Chair: Kenneth Wong - Brown University
- Room C -

Teacher Quality Policy When Supply Matters Jesse Rothstein � University of California at Berkeley and NBER

Moving High-Performing Teachers to Low-Performing Schools: A Randomized Experiment
Steven Glazerman, Ali Protik, Bing-ru Teh and Julie Bruch � Mathematica Policy Research

Portability of Teacher Effectiveness Across School Settings
Zeyu Xu, Umut Ozek and Matthew Corritore � American Institutes for Research

Teacher Mobility and Student Learning
Gianna Barbieri � MIUR; Claudio Rossetti � LUISS; Paolo Sestito � Bank of Italy

Meritocracy for Teachers: Evidence from Colombia
Alejandro Ome � University of Chicago

18:30-19:00 Reception with Light Hors d�Oeuvres
Speech by Raj Chetty, Harvard University (on video)
Friday, October 5, 2012
9:00-10:30 Rapporteur Session
Chaired by Douglas Besharov, University of Maryland; Association for Public Policy Analysis and Management

Rebecca Maynard, University of Pennsylvania
Gianluca Argentin, University of Milan-Bicocca
Aline Pennisi, Ministry of Economy and Finance
Erich Battistin, Istituto di Ricerca per la Valutazione delle Politiche Pubbliche
10:30-12:30 Panel Session: "Translating Research Results into Practice"
Chaired by Paolo Sestito, Istituto Nazionale per la Valutazione del Sistema Educativo di Istruzione e di Formazione; Banca d�Italia

Francesco Profumo, Italian Minister of Education
Ugo Trivellato, University of Padova
Andreas Schleicher, Organisation for Economic Co-operation and Development (on video)
Pina Antonaci, Istituto "De Pace" di Lecce
12:30 Closure


With additional support from

Cooperating agencies