A Game of Trophies

This blog co-written with ValeriaZhavoronkina, Alessandro Martinello and Daniel Safai.

That special time of year draws near, where children huddle close to their parents by the fire and where virtuous, respectable econ students sleep with their copy of Wooldridge under their pillow. This week, theEconometric Gameis held in Amsterdam, from March 31st to April 2nd. On thisoccasion, let us think back;On Thursday the 17th of April, 2014, a clash of titans took place in Amsterdam and ultimately left the University of Copenhagen bringing home the Econometric Game Trophy to its rightful home. Here follows the epic tale of the frightful events that preceded the triumph.

A Game of Trophies

Rain was pouring mercilessly down on Bertel, Captain Commander of the Night Watch, as he saluted the four young economists standing straight before him. Winds from Beyond the Great Belt brought him the familiar, terrifying scent of death and decay he learned to recognize and fear. The powers of the White Walkers of Aarhus were rising, and the small group of warriors summoned here was his best hope for the ultimate battle of good and evil that lay ahead; The Econometric Game. Gathering around the great tree in the center of the courtyard, the four warriors bellowed their vows as one;

Night gathers, and now my watch begins. It shall not end until my death. I shall take no freshman, hold no internship, supervise no undergrad. I shall wear no business attire and win no drinking competitions. I shall live and die at my screen. I am the econometrician in the darkness. I am the programmer on the walls. I am the shield that guards the realms of men. I pledge my life and honor to the University of Copenhagen, for this night and all the nights to come.

As ominous clouds drifted toward dusk, the four warriors were riding forth to ancient Amsterdam, whose walls already bore the marks left by the scourge of the White Walkers of Aarhus.

Day 1

On Tuesday, the four warriors arrived at the battlefield in the University of Amsterdam. There, the noble Professor Pradho Menhan and the Order of the Jury divined that the battle was to happen at the Development Economics field, where the 29 teams summoned to counter the White Walker menace would fight to their last breath for two days to deliver the best prediction of poverty rates in developing countries. The teams surviving the fierce battle of day 2 would have little rest before clashing their weapons again in the Final Battle of day 3.

The four brave heroes spent the day whetting their blades and praying to the Gods of the Stata core development group.

Day 2: The First Battle

The teams were watering their steeds and optimizing caffeine blood levels when the hotel dining hall went quiet and coffee froze as the White Walkers of Aarhus entered the hotel hall, their dead blue eyes scanning the competitors. Quickly, the human teams left the hall and headed straight for the battleground. As the four warriors of Copenhagen made their entrance, the smell of iron, blood and overclocked computers was thick in the air. The teams once again went quiet as the ancient Order of the Jury entered and distributed the mighty usb sticks - weapons of mass poverty indicators.

From left: Alessandro, Valeria and Daniel

The Case

The dataset contained information from household surveys done in Indonesia for the years 1996, 1999 and 2002. The variable of interest (the y) was consumption, and it had been removed from the 2002 data. Poverty is defined as the fraction of households falling below the given poverty line. The goal was to predict the poverty rate in 2002 using other poverty-related measures that are functions of consumption. A first simple attempt at answering the research question would be to estimate the regression of consumption on explanatory variables for the 1996 and 1999 data using OLS and use the coefficients to impute consumption using the available x-variables for 2002.

The central idea of the Copenhagen team was to focus on two different sources of error in this simple estimation equation;

1. Measurement error. A problem with consumption surveys is that respondents who produce their own food and goods can give particularly imprecise answers. This implies that the signal-to-noise ratio is especially low for poor households. Among the richer households, measurement error might be a lesser concern.
   
2. Omitted endogenous regressors (or non-constant parameters). To predict the poverty rate, it is important to minimize the estimation error for the households that are close to the poverty line. A small error around the poverty line can tip a household on both sides of the poverty line. If the true model parameters vary across different households, then we focus on getting the right parameters for those households that are close to the poverty line.

The team proposed a Weighted M-estimator (inspired by the inverse probability-weighted estimator), where the weighting function was constructed to explicitly focus on the parts of the sample that would be most informative to avoid these types of errors. To deal with errors of type 1, the estimator applied a higher weight on the richer households. To deal with errors of type 2, the estimator applied a higher weight on households close to the poverty line. Both estimators outperformed the baseline OLS results.

The Red Wetting

As the end of the battle drew closer, the fighting intensified. The air over the Copenhagen table abounded in high-fives, sparks flying from overheated CPUs, and Italian curses from the team's Alessandro of house Martinello (three times regional swearing champion in Venice). In the distance, foul howls rose above the buzz of cooling vents as the Dublin team unleashed their golden direwolf (or rather, the rump of a direwolf, cf. the image below).

After the battle, the four exhausted heroes hoisted their weapons back on their shoulders and plodded to the communal serving of victuals. Unleashing brutal politeness, all contestants sheathed their laptops and engaged in light conversation while passing around salt and bread. Later that evening, frothy ale flowed like the sea and music lulled everyone into a state of unpreparedness... until suddenly the music broke into a dark, eerie tune, and judgment was passed. The details of that terrible night are lost in rumors and tears. With festive music against the background, 20 out of the 30 participating houses were brutally slaughtered with their hands still gripping mugs --- the floor turned wet with beer and blood!

Day 3: The Final Battle

None of the surviving teams uttered a word the morning after the Red Wetting. The heroes recognized many famed and honorable houses as they approached the grounds of the Final Battle: the stern Lannisters of Harvardly Rock in their crimson and gold, House St. Gallen of the Eyrie, House Carlos III of Madrid with their sun and spear coat-of-arms. Beyond, the fierce warring tribes of the Warsaw plains exchanged war cries with the Tyrells of Tilburg and house Hightower of Rotterdam.

Their hearts all sank, however, when the White Walkers of Aarhus stepped on the battlefield, their icy armour unscathed by the fierce combat of the day before, and untiringly marching forward, now so close to victory that their icy gaze was almost unbearable for the humans.

The Case

The teams received an additional dataset that contained a different measure of consumption for a different (random) sample, available for all years. This measure was aggregated at a higher level and cheaper to obtain (e.g. instead of querying about single food items, the survey asked for gross consumption of food). However, it quickly became clear that this measure was considerably more noisy than the other and tended to underestimate consumption on average.

The question was: can you use this measure to improve your prediction of poverty measures for 2002? The econometric challenge was that only one of the two measures was observed for each household. If both measures had been observed, one could have regressed the precise y on the imprecise y, as well as x'es, but this was not possible.

The Night Watch team of Castle CSS crafted a two-pronged attack for the day's case. First, the bad-mouthed but strong-hearted sourthern warrior of the team, Alessandro, summoned the fiery Gods of the South before spewing sparks and curse words while reformulating a GMM measurement error model framework to identify the relative non-classicality of the measurement errors in the new, noisy data. He revealed a high degree of non-classicality in the dataset, which invalidates most traditional microeconometric approaches.

Second, as the renegade faceless woman Valeria held the flanks and protected the rest of the ream, the brave protagonist, Daniel, along with the fast-typing team captain and primary provider of poletter, Anders, surprised all other teams with an original approach to the problem that no one expected.

This solution was inspired by comparing the distribution of the true and predicted consumptions;

This showed that distribution of predicted y's was much tighter than the actual distribution. Thus, the fraction of households falling below the vertical poverty line, x=0in the graph, was substantially underpredicted. As the main objective was to predict poverty, the idea of the team was to fit (intuitively “inflating”) the distribution of predicted y's so that it would match the observed y's. This is normally not a preferred approach since the econometrician is concerned about predicting each individual's y, but since the goal was to predict the amount of poor individuals and not precisely who is poor, it was appropriate for this task.

The fit was obtained by combining information from the observed distribution of the noisy consumption measure with basic OLS predictions of the precise consumption measure.

The Spoils

After the battle, each of the teams in the finals uncovered their tactics in 5-minute demonstrations. It quickly became evident that most of the teams had chosen a direct mode of attack, making use of a matching approach. The Copenhagen team was the only one to enter the city through the back gate and avoid the measurement error traps.

At what seemed an eternity after the final feast, the honorable Order of the Jury returned from the decision chamber and stood atop a stage, looking down upon the battle-weary troops. A gasp cut through the air when the White Walkers of Aarhus were granted third place, shared with the local House Baratheon of Amsterdam. The Lords of the Bristol Crossing were awarded silver medallions.

The hall turned silent. Nothing could be heard except for the faint sound of a lucky coin turning in the hand of a certain badmouthed southerner. And the winners are --- Copenhagen! The hall erupted with merriment, high fives, and cries of victory as the trophy was once again in its rightful hands! Alas, as the Night Watch team from Copenhagen turned to the White Walkers of Aarhus, the ghouls turned into joyous human beings, gulping down mead and sharing stories of their sweet daughters. Their human spirits had been freed from the their icy prisons by the warming force of the good, sound econometric spirit that was in the room around them.

From left: Valeria, Anders, Daniel and Alessandro

Daniel and Alessandro