Guiding Questions For Seminar Preparation

Here is a list of questions that we think might help you get started with preparing for the seminar (and, implicitly, the coursework). This is not an exhaustive list, it simply provides a starting point for discussion and reflections. Note that there is a “General Questions” and a “Seminar 1 Questions” section. The general questions are meant to be fairly paper-agnostic. That means that for some papers, some of the questions might have extremely overlapping answers, or some of the questions might not make sense at all. It’s the sacrifice we had to make in the name of generality :). Try to think about these questions (and ideally write down your answers before the seminar or at least annotate the paper based on these questions). Write down any other thoughts as well and bring your notes to the seminar. The more you contribute to the discussion, the more you learn.

General Questions

Big Picture

1. What is the context in which the paper was written? (Think about, or find in the paper if mentioned, what works preceded the paper, what people believed at the time, what other related work was happening at the time)
2. What is the research question?
3. What is the motivation? (i.e. why is it an important research question to consider)
4. What are the claimed contributions?
5. What are the core concepts around which arguments are built? Are these well-defined?

Experiments

1. What are the core experiments?
2. What do the authors claim these show? Are the experiments directly backing up the claims?
3. What assumptions do they make? Are these explicit or implicit? How constraining are they?
4. What are the main factors (variables) that impact the experimental results? Are these well-controlled?
5. What are the experimental details needed to assess the validity of the results? Are all of these specified?
6. What are the limitations of the experiments? Are these limitations affecting the validity of the results?
7. Are the experimental results statistically significant?
8. Are any of the results (especially the qualitative ones) cherry-picked? I.e. Are the presented results reflecting what is observed in expectation or did the authors choose to only present results that don’t contradict their narrative?

Back to Bigger Picture

1. Do the experiments show what the authors claim they show? Is this adequately reflected in the claimed overall contributions?
2. What other experiments would have strengthened the claims? Are the claims still valid without these experiments?
3. Are there any unjustified claims throughout the paper? (Here think outside of the claimed contributions)
4. (More towards extension:) If the paper is building on existing methods, are the methods actually doing what the authors claim they are doing?
5. Are there any obvious reasons why this paper might not be reproducible (e.g. think of data availability, computational constraints, availability of all core experimental details)?
6. Are there any ethical implications of this paper? What is the broader impact of this work? Are there any societal implications?

Extensions

1. What are the wider technical implications of the paper?
2. What are intriguing results the authors did not explore further?
3. What other subdomains relate to aspects of this work?
4. Is controlling for variables the authors did not consider likely to tell a different story?
5. What alternatives are likely to be interesting that the authors did not consider? What would these tell us and why is that interesting?

Additional Questions for Seminar 1

1. How are shape and texture defined in this paper?
2. Why should we care about establishing if the texture or the shape hypotheses are right beyond pure intellectual curiosity?
3. What does this paper tell us about anthropocentric interpretations of neural networks?