Conducting Panel Data Analysis

For a dataset with entities id observed over time and outcome y, regressors X:

STATA
* 1. Declare panel
xtset id time
xtdescribe   // check balanced/unbalanced, N and T

* 2. Baseline models
reg y X, vce(robust)                      // Pooled OLS
xtreg y X, fe                             // Fixed Effects
xtreg y X, re                             // Random Effects

* 3. Model selection
xttest0                                   // Breusch-Pagan LM: RE vs POLS
hausman fe_est re_est                     // FE vs RE

* 4. Diagnostics (on FE model)
xttest3                                   // heteroskedasticity (Wald)
xtserial y X                              // serial correlation (Wooldridge)
xtcsd, pesaran abs                        // cross-sectional dependence

* 5. Final robust specification
xtreg y X i.time, fe vce(cluster id)      // or xtscc if CD present

In R: plm::pdata.frame(), plm(y ~ X, model="within"/"random"), phtest(), pbgtest(), pcdtest(), coeftest(.., vcov=vcovHC).

Progress:
- [ ] Step 1: Structure and declare panel; classify short vs. long, balanced vs. unbalanced
- [ ] Step 2: Run EDA — within/between variation, VIF, CD test, unit root tests
- [ ] Step 3: Estimate POLS, FE (+ time effects), RE as baseline candidates
- [ ] Step 4: Run Chow/F-test, Breusch-Pagan LM, Hausman (or Mundlak) to select estimator
- [ ] Step 5: Test residuals for heteroskedasticity, serial correlation, CD; apply correct SEs
- [ ] Step 6: Address endogeneity (GMM/IV) and run robustness checks

Step 1: Structure and Declare

• Convert wide → long format (one row per entity-time pair).
• xtset id time (Stata) or pdata.frame(df, index=c("id","time")) (R).
• Classify panel type:
  • Short/micro (large N, small T): standard FE/RE apply.
  • Long/macro (small N, T>20): requires panel unit root/cointegration methods — flag this early, it changes the entire toolkit.
• Check balance: xtdescribe. If unbalanced, confirm missingness is not survivorship-driven (document this assumption in the paper).

Step 2: EDA and Pre-Estimation Checks

• Report overall/between/within standard deviations (xtsum). If a key regressor has near-zero within-SD, warn that FE cannot identify its effect — this is a critical finding to report, not skip.
• Correlation matrix + VIF from a preliminary Pooled OLS (estat vif). Flag VIF > 10 (or > 5 in stricter fields).
• If T > 15–20: run Pesaran CD test first, then choose unit root test accordingly:
  • CD absent → Levin-Lin-Chu or Im-Pesaran-Shin (first-generation).
  • CD present → Pesaran CIPS (second-generation).
• Skip unit root testing for short panels (T too small for asymptotics to hold).

Step 3: Specify and Estimate Candidate Models

Estimate all three as a comparison table (standard practice for peer review):

• POLS: baseline column, assumes u_i = 0.
• FE: time-demean or LSDV; add year dummies (i.time) to absorb common shocks. Note time-invariant variables get dropped — flag this explicitly if the paper's hypothesis involves one.
• RE: GLS estimator; retains time-invariant predictors but requires Cov(u_i, X_it) = 0.

Step 4: Model Selection Tests

Run in this order and report all three, even if the answer seems obvious:

1. F-test (Chow): FE vs. POLS. p < 0.05 → reject POLS.
2. Breusch-Pagan LM: RE vs. POLS. p < 0.05 → reject POLS.
3. Hausman: FE vs. RE. p < 0.05 → use FE.
   • If errors are heteroskedastic, standard Hausman is invalid — use Mundlak's approach (add entity means of time-varying regressors to RE model) or xtoverid (Sargan-Hansen).

Decision rule: if both F-test and LM reject POLS, the question becomes FE vs RE via Hausman. If Hausman is inconclusive or invalid, justify FE on theoretical grounds (omitted variable bias is usually the safer default for causal claims).

Step 5: Post-Estimation Diagnostics

Test the chosen model's residuals:

• Heteroskedasticity: Modified Wald test (xttest3 for FE).
• Serial correlation: Wooldridge test (xtserial).
• Cross-sectional dependence: Pesaran CD test on residuals (xtcsd, pesaran).

Apply corrections based on findings:

Never report uncorrected SEs if any test rejects the null — this is a common reviewer rejection point.

Step 6: Endogeneity and Robustness

• Lagged dependent variable present → standard FE is biased (Nickell bias). Use Difference GMM (Arellano-Bond) or System GMM (Arellano-Bover/Blundell-Bond).
  • Report AR(2) test: need p > 0.05 (no second-order serial correlation).
  • Report Hansen/Sargan test: need p > 0.05 (instrument validity). Also report instrument count relative to N (too many instruments weakens Hansen test power).
• Reverse causality in static model → FE-2SLS with external instruments; report first-stage F-stat (>10 rule of thumb for instrument strength).
• Robustness section: re-estimate with alternative proxies, sub-samples, and winsorized data (1st/99th percentile) to show coefficient stability. Present as a side-by-side table against the main specification.

Example 1: Input: Firm-level panel, N=500 firms, T=8 years, testing effect of R&D spending on profitability. Output: Short panel → skip unit root tests. EDA shows R&D/sales has substantial within variation (good for FE). VIF all < 5. F-test rejects POLS (p<0.001); Hausman rejects RE (p=0.002) → FE selected. Wooldridge test detects serial correlation (p=0.01) → use xtreg y X i.year, fe vce(cluster id). Robustness: re-run with winsorized R&D variable and sub-sample excluding tech firms; coefficient sign and significance hold.

Example 2: Input: Country-level panel, N=30 countries, T=25 years, testing effect of trade openness on GDP growth. Output: Long panel → run Pesaran CD test first (CD present, common macro shocks). Use CIPS unit root test; GDP and openness are I(1), check cointegration (Pedroni/Westerlund) before levels regression. Given CD, use Driscoll-Kraay SEs regardless of FE/RE choice. Hausman invalid due to heteroskedasticity → use Mundlak approach, which favors FE-equivalent specification.