How to read papers? The three-pass approach (see original post)

First pass (5 min) - Bird’s-eye view or “the big picture”

  • Skim through the structure.
  • Read abstract, title, introduction, and conclusion.
  • Is the paper helpful? If not, discard.

Assessment: category, context, correctness, contributions, and clarity

  • Category
    • Prototype?
    • Literature review?
    • New optimization method?
  • Context
    • Put the paper in perspective: what other papers are related?
    • Semantic tree: Is the paper an important branch or an uninteresting leaf?
  • Correctness
    • Are the assumptions valid?
  • Contributions
    • Meaningful?
    • Useful?
    • Novel?
    • What problems does the paper solve?
  • Clarity
    • Is the paper well written?
    • Grammar mistakes or typos?

Reasons to stop reading

  • You lack of background information.
  • Not interesting/useful.
  • Poorly written.
  • Unfounded/false assumptions.

Second pass (1h) - Read all but details

  • Read paper as a whole.
    • Do the tables and figures make sense?
    • Sloppiness (mistakes or discrepancies in axes, captions, labels, etc.) may indicate overall bad quality.
  • Ignore details such as proofs & equations.
  • Make notes, write down key points.
    • Mark relevant references to build your semantic tree.
  • Rephrase key points.
  • Stop reading if:
    • Paper is not useful.
    • You need to read background material first.

Third pass (1h-5h) - Read all

  • Caution: only do if paper is worth it!
  • Read complete paper (must do for reviewing).
    • Identify and challenge every assumption in every statement.
  • Try to re-implement the results.
    • Re-implement steps using flowcharts, pseudocode, programming language (e.g., Python, Jupyter notebooks).
  • Upon completing this step you:
    • Should know the paper’s strong and weak points, such as
      • missing citations and
      • potential issues.
    • Can reconstruct the paper’s structure and explain it in simple terms.

Productivity tips

Highlighting

  • Content
    • Yellow: Interesting/important citations
    • Green: Concept definitions.
    • Purple: Takeways, rephrased sentences for direct reference.
    • Red: Hard to understand.
    • Orange: Interesting/important references to snowball.
  • Form
    • Blue: Word definitions, synonyms, pronunciation.
    • Gray: Interesting word choices, sentences for academic writing.

Three-pass approach for literature review

First pass - Collect useful papers

  • Type keywords and find 3 to 5 recent papers.
  • Classify according to year and the count of citations.
  • Cluster papers by topic.

Second pass - Snowballing

  • Try to spot recent work from repeated authors (visit their website).
  • Download commonly cited papers and apply the three-pass approach.

Third pass - Search for novel work in the field

  • Visit the websites of top conferences/journals and look through the recent proceedings.
  • Identify related high-quality work.

Further reading

  • Keshav, S. (Srinivasan)(2007). How to read a paper. ACM SIGCOMM Computer Communication Review, 37(3), 83–84. (URL: https://dl.acm.org/doi/pdf/10.1145/1273445.1273458)
  • Cormode, G. (2009). How not to review a paper: The tools and techniques of the adversarial reviewer. ACM SIGMOD Record, 37(4), 100–104. (URL: https://dl.acm.org/doi/pdf/10.1145/1519103.1519122)
  • Meier, A. (1992). How to review a technical paper. Energy and Buildings, 19(1), 75–78. (URL: https://eta-intranet.lbl.gov/sites/default/files/how-to-review-a-technical-paper_0.pdf)
  • Roscoe, T. (2007). Writing reviews for systems conferences. (URL: https://www.cl.cam.ac.uk/teaching/1011/R01/review-writing.pdf)

Tips for better understanding

Feynman tecnhique (see original post)

  • Identify the subject
    • Write down everything you know about the topic.
  • Teach it to a child (no prior knowledge)
    • Write everything you know about the topic.
      • Speak in plain terms (no jargon, vebose, clunky language).
      • Be brief (assume short attention span).
  • Identify your knowledge gaps
    • Go back to the learning material to clarify weak points in the explanation.
  • Organize + Simplify + Tell a story
    • Take off technical terms and complex language.
    • Find the essential kernels.
    • Practice reading your story out loud;and check when
      • it stumbles = incomplete toughts, or
      • stop being simple.

Mindmaps

  • Title in the center.
  • First-level nodes: main sections.
  • Second-level nodes: subsections.
  • Leaves: information.

Time-boxing (Parkson’s law + Pareto principle)

Constrain your reading to a predetermined time span and focus on the main points.

“Work expands so as to fill the time available for its completion”

— Cyril Northcote Parkinson

“For many events, roughly 80% of the effects come from 20% of the causes.”

— Vilfredo Pareto

Rubber ducking (from the The pragmatic programmer)

  • Explain code to other person (rubber duck, potted plant, teddy bear) step by step (while he nods up and down).

References

The LaTeX helpers

Pretty tables

https://people.inf.ethz.ch/markusp/teaching/guides/guide-tables.pdf

Latex formatting

https://c.albert-thompson.com/latex-pretty/

Bibliography formatting

https://www.bibtex.com/format/

Tidy-up bibtex

https://flamingtempura.github.io/bibtex-tidy/

Get a professional account on Overleaf via TU Delft

https://www.overleaf.com/edu/tudelft

Integration with Github

Download Github desktop at https://desktop.github.com/ Comprehensive course: https://www.coursera.org/learn/introduction-git-github

Writing tips

Academic writing courses

  • Writing in the Sciences by Dr. Kristin Sainani, Stanford University (Coursera)
Paragraphs

Books

  • The Elements of Style. William Strunk, Jr.

Use passive voice purposefully and sparingly

Where to use passive voice? In the methods section. Why? Because what was done is more important than who did it. Besides, the methods section is not the most exciting prose of the paper; most people just skim this section. It is very tricky to write the methods section in the active voice because you have to be quite creative to avoid starting every sentence with we. Hence, it does not need to be as lively as the other sections of the paper.

Sentence starters, transitionals, and other useful ords

http://www2.eit.ac.nz/library/ls_guides_sentencestarters.html

Referring to other literature reviews

  • The readers may refer to Elshaer and Awad (2020) for a comprehensive review on metaheuristic for VRP. This review focuses on the machine learning assisted VRP algorithms and Section 4 discusses all the relevant papers.
  • For a comprehensive review of robust optimisation, we refer interested readers to (Bertsimas, Brown, and Caramanis 2010).
  • A good overview of such algorithms give Molenbruch et al. (2017), who note that “in the standard problem, operational costs are minimized, subject to full demand satisfaction and service level requirements”, which relates closely to our study presented below.

Introducing methods

  • This section provides an overview of various VRP modelling methodologies that are supported by data analytics and machine learning. In particular, we focus on the modelling techniques for handling uncertain, incomplete, imprecise or ambiguous data in VRPs, including stochastic programming, robust optimisation, chance constrained programming and data forecast.

Proofreading

https://www.grammarly.com

How to describe table fields?

Check how table columns are described in the text in the following papers:

How to forward redears to other papers?

  • Details on the methods used to operationalize this controller have been extensively documented in several publications (Oh et al., 2020a, Nahmias-Biran et al., 2019, and Basu et al. 2018), to which we refer the interested readers.
  • The synthetic population for 2030 (6.7 million individuals) was generated using a Bayesian approach (Sun and Erath, 2015) based on socio-economic data, land-use data, and relevant control totals (see also Zhu and Ferreira, 2014 for more details on the population synthesis).
  • The calibration and validation also included matching simulated outputs to observed screen-line counts, public transit smart card data and network travel times (for more details on the model calibration, the reader is referred to Oh et al., 2020a).
  • The readers may refer to Elshaer and Awad (2020) for a comprehensive review on metaheuristic for VRP. This review focuses on the machine learning assisted VRP algorithms and Section 4 discusses all the relevant papers.
  • For a comprehensive review of robust optimisation, we refer interested readers to (Bertsimas, Brown, and Caramanis 2010).

Introducing methods

  • This section provides an overview of various VRP modelling methodologies that are supported by data analytics and machine learning. In particular, we focus on the modelling techniques for handling uncertain, incomplete, imprecise or ambiguous data in VRPs, including stochastic programming, robust optimisation, chance constrained programming and data forecast.

Literature review examples

Academic writing tips

https://trumpexcel.com/data-entry-form/ ### Evaluation

  • MME Literature Assignment Description 2020: https://brightspace.tudelft.nl/d2l/le/content/43739/viewContent/1957973/View
  • MME Literature Assignment Grading Rubric: https://brightspace.tudelft.nl/d2l/le/content/43739/viewContent/1958251/View

You may also enjoy

Filter Used References Latex

less than 1 minute read

When working on a topic, it is common to cite a group of seminal references across several papers.

Pytest

less than 1 minute read

Pytest