Research design connects a research question to the evidence needed for an answer. It specifies what you will observe, compare, build, or test. It also sets the limits of the conclusion.
A design can change as the project develops. Record each material change and explain why the available evidence required it.
Start with the intended claim
Write one sentence describing the conclusion the study should support. The sentence can remain provisional, but it must be specific enough to test.
Example:
Under historical order patterns and the current warehouse layout, the proposed batching policy reduces median picking distance without increasing the share of late orders.
The sentence identifies the setting, intervention, comparison, outcomes, and boundary. It also exposes the evidence the study needs.
Connect the question to the design
For each research question, complete the following chain:
| Design decision | Question to answer |
|---|---|
| Unit of analysis | What object, period, person, order, route, or instance is observed? |
| Evidence | Which data or observations represent the phenomenon? |
| Comparison | What baseline, counterfactual, or alternative supports the claim? |
| Analysis | Which calculation, model, or qualitative procedure produces the result? |
| Decision rule | Which result would support, weaken, or reject the claim? |
| Boundary | Where should the conclusion apply? |
If one link remains vague, the research question is not ready for final data collection or implementation.
Define the unit of analysis
The unit of analysis determines what one observation represents. It may be a customer order, route, shift, warehouse, interview, company, or benchmark instance.
Use a unit that matches the research question. A question about daily service performance needs day-level evidence. A question about order-level lateness needs order-level evidence. Aggregating too early can hide variation and create an unsupported conclusion.
State the population, sample, inclusion rules, exclusions, and observation period. Explain how missing values and unusual cases will be handled.
Choose the evidence
List the evidence needed for each part of the claim. Sources may include system logs, interviews, process observations, documents, experiments, simulation, and published benchmark data.
Check 4 properties:
- Relevance: the evidence measures the concept named in the question.
- Coverage: the sample includes the important situations and operating conditions.
- Quality: the measurements are accurate enough for the intended analysis.
- Access: the data can be obtained, used, stored, and reported under the applicable agreements.
Document how each dataset was produced. A clean table can still contain biased measurements or information recorded after the decision being studied.
Select the comparison
A result needs a reference. Depending on the study, compare the proposal with the current process, an established method, a control group, a prior period, or an exact result on a reduced case.
Make the comparison fair. Use the same inputs, information, time budget, and outcome definitions unless the research question explicitly studies one of those differences.
For company projects, reproduce the current procedure before testing the new one. A weak or inaccurate baseline can overstate the contribution.
Decide how the evidence will be analyzed
Name the procedure and explain why it can answer the question. Possible choices include statistical analysis, optimization, simulation, case comparison, thematic analysis, or a mixed-method design.
Separate the method from the methodology. The method is the procedure. The methodology explains why the procedure and evidence can support the intended claim. See Method versus methodology.
Define the primary outcome before examining the final results. Then identify secondary measures, subgroup analyses, and sensitivity tests.
Check validity threats
Every design has limits. Identify the most plausible threats and decide how to reduce or report them.
| Threat | Example | Response |
|---|---|---|
| Measurement | Recorded processing time excludes waiting | Compare system logs with direct observations |
| Selection | Test days omit seasonal peaks | Include peak and normal periods |
| Confounding | Demand changed after the new policy began | Replay both policies on the same demand scenarios |
| Implementation | Baseline code is weaker than the real process | Validate it with company planners and known outcomes |
| Generalization | One warehouse has an unusual layout | State the boundary and test varied synthetic layouts |
Focus on threats that could change the conclusion. A long generic limitations list does not replace a response to the main risks.
Plan the analysis before the final test
Write the analysis plan while changes are still cheap. Include:
- datasets and inclusion rules;
- baseline and proposed methods;
- primary and secondary measures;
- configuration and computational budgets;
- sensitivity tests and reporting tables.
Reserve separate data or instances for final evaluation when method development involves tuning. Freeze the main configuration before running the final test.
Minimum design record
Before implementation or intensive data collection, prepare a 1-page design record containing:
- the problem statement and research question;
- the intended claim and unit of analysis;
- the evidence sources and access conditions;
- the comparison and analysis procedure;
- the main validity threats;
- the decision rule for interpreting the result.
Review the record with the supervisor and company contact. Update it when the research question, data, model, or evaluation changes.
Continue with Scope and limitations to define the study boundary. Optimization projects should also use Formulating an optimization problem and Benchmarking optimization methods.