Week 25: Plan for new interactive model

Last week, I wrote about our new plan to write a visualization paper based on the IQP team's ranking application. At conferences like VAST, there are several different types of papers one can submit, and our first challenge was deciding where to pigeonhole our application. After talking with Professor Harrison, a notable vis expert in our CS department, we determined that the novelty in our project is the user interface. As such, our paper would fall into the category of systems or design study. According to Processes and Pitfalls in Writing Information Visualization Research Papers, a sort-of guidebook to the world of writing vis, systems papers focus on architecture choices and the design of abstractions. Design study papers, in contrast, demonstrate a new visual design to solve a problem, often using existing algorithms or techniques. If we consider ours to be a systems paper, we would probably focus on explaining the pipeline (from choosing a dataset to building a ranking from partial information to then explaining the learned ranking in a table). If we consider it a design study paper, however, we would talk about the choices we made for the drag-and-drop build stage and the results of the user study.

Professor Harrison suggested making some modifications to the existing application to highlight some of the active learning mechanics that the application is supposed to facilitate. These modifications should make the paper more enticing for reviewers to accept.

I made a couple mockups of the changes we would like to make. Here are a couple screenshots of the current application:
Original Build view, list view is shown.
Original Explore view (page 2), this bar chart shows the weights of the attributes learned by the ranking.
In the new mockup, the bar chart in Explore serves a new purpose. Instead of showing the weights of the learned ranking, it now is used in Build to show the values of the attributes of any one object. In this way, the user can hover over an object and see what information is known about it. The hope is that this will help the user understand what the program is using to build the ranking without giving them enough of the raw data to bias their intuition.
New Build view, not hovering on any one object, so bar chart is empty.
Same view as previous, imagine mousing over one of the states.
Another new feature would be a progress bar that shows the user how confident the model is that the ranking reflects what the user wants. Confidence will be calculated using some existing active learning algorithm, and the progress bar would go to 100% as confidence increases.
Progress bar. With this new feature, the ranking is re-calculated every time the user adds a new object.
The last change would be to modify the table in Explore to show the user the relative weights of the attributes and how each attribute really contributes to the model. This will closely mimic the view created by the Podium group.
Explore view with a different looking table. The bar charts for each attribute are colored to match the attributes in the Build view bar chart for continuity.
These changes aren't very revolutionary, but we only have a couple weeks to implement them before submission. Caitlin and I are hoping to recruit members of the IQP team who are masters of the codebase and can hopefully make these changes quickly.


Comments

Post a Comment

Popular posts from this blog

Week 31: Datasets for fairness

This week, Diana and I looked for some datasets on which to evaluate our fairness criteria. This is no simple task. For one, the datasets need to be large enough to be split into training, calibration, and testing sets. Second, there needs to be some protected group attribute like sex, race, or age. Third, there needs to be some outcome ranking as well as some truth value. The truth is often the hardest attribute to find. In this post, I introduce some of the datsets we decided to use. I try to identify each of the necessary features that we need to properly test our fairness criteria. COMPAS The COMPAS dataset is an obvious choice because it is well ingrained in the literature about fairness. It has 18,000 observations (n=18000) and 1000 attributes (k=1000). Protected groups can be sex (81/19) or race (51/49). The outcome attribute can be the recidivism score, the score used in court that represents the risk that the accused will commit another crime. The truth attribute is a li...
Read more

Week 32-33: Wrapping up

Hi all. This is my last blog post for this project. The last few weeks, we spent little time working on the project and instead focused on finishing up our classes. To be honest, we lost some motivation after finishing the paper, and we've been reluctant to start up new tasks so close to the end of the year. In this post, I'd like to talk about some of the things I've learned in this project other than what we found in our research. This post is really more a service to myself; to help me reflect on this past year and what I've learned. When I started this project, I didn't understand how research in computing worked. My exposure to research had been more biology/clinical, in which research involved setting a hypothesis about some protein, designing an experiment with positive and negative controls, and then running the experiment repeatedly until confident in the results. In computing research, my experience has been of two varieties: reading papers and proce...
Read more

Week 30: Returning to Fairness

With the LineUp paper submitted, it's now time to return to the fairness problem we were working on before. As a bit of a refresher for you and for me, I'll review what we've done with fairness so far and what remains to be done. Fairness in Ranking Up until now, the literature regarding fairness has focused on fairness in classification. As an illustrative example, consider fairness in hiring practices with repect to applicant sex. A job opening might receive 100 applicants evenly split into 50 males and 50 females. If males and females are equally capable of doing the job, then you might be concerned to see the hiring company mostly interviewing male applicants. To evaluate fairness, you would compare each applicant's capability (capable of performing duties, not capable) to their outcome (interviewed, not interviewed). In a completely fair scenario, the same proportion of capable applicants from each sex group would receive an interview. Small deviances from this...
Read more