Week 20: Experimental plan

Hello! Full disclosure, last week's blog post contains information on some of the things I did this week. I wrote it with the conclusion in mind, so (I think) it sounds pretty well thought-out. However, last week's work did not go as smoothly as I wrote it to sound. For one, after I collected 11 different datasets on which to test our models, Caitlin realized that we weren't collecting the right kind of data. The datasets I chose didn't have a clear outcome variable (which I tried to write about in last week's post), but we definitely need one to test the models (or so we think). That wrench in our plans led Caitlin and I to having a long discussion about how the ranking actually works. We drew lots on the whiteboard, and then I went home and tried thinking through the process myself. The rest of this post is a walkthrough of my logic steps as to how an experiment like this might work. I started by thinking about scoring in the realm of statistical parity. Stati...