A Survey of Web Application Security Tutorials
Bhagya Chembakottu · Martin P. Robillard
Developers rely on online tutorials to learn web application security, but content quality is inconsistent. We reviewed 132 free security tutorials to investigate their origin and what they contain. Most tutorials originate from vendors and focus on marketing. Few provide runnable code examples or direct links to authoritative resources (such as Open Web Application Security Project, Common Weakness Enumeration or Common Vulnerabilities and Exposures).

Tutorial size by type

We computed text length from the visible textual content of each tutorial after removing HTML elements (e.g., tags, scripts, navigation components) and stripping code blocks. We used a simple whitespace-based tokenization.

Tutorial type vs. content organization

We categorized each tutorial by its overall tutorial type and its content organization structure. Tutorial type captures the general nature of the tutorial. Content organization describes how the tutorial structures information. We then counted the number of tutorials that fall into each combination of tutorial type (rows) and content organization (columns). Each cell in the heatmap represents the number of tutorials in that category pair. Hovering over a cell reveals the exact count.

Security standards referenced across tutorial categories

We coded whether each tutorial referenced common security standards and taxonomies (e.g., OWASP, CWE, CVE, NIST, ISO 27001). This chart summarizes how often each standard is referenced within each tutorial category.

Statistical analysis

This section visualizes key statistical and descriptive results. Each subsection pairs an optimal visualization recommendation with the exact figures and test results reported in the study.

Text Length Distributions (Mann–Whitney U Tests)

Text length is heavily right-skewed, making medians and spread more informative than simple means. The box plot compares the narrative word count of tutorials with "Code included" versus those with "No code". It visually explains why the difference in average word counts was not statistically significant after adjusting for multiple comparisons.
Descriptive Stats: Mean ≈ 2,500 words; Median ≈ 1,750 words (right-skewed distribution).
Test Results: Mann-Whitney U = 1001.5, p = 0.0416 (Not significant after Bonferroni correction threshold of α = 0.0125).

The Rarity of Code Examples (Descriptive Statistics)

Code examples were highly uncommon in the dataset. A horizontal bar chart or a waffle chart effectively highlights the stark contrast between the vast majority of narrative-only tutorials and the very few that provide actionable, runnable code.
Distribution: No Code 106 (80.3%), Short Snippet 20 (15.15%), Runnable Example 3 (2.3%), Pseudocode 3 (2.3%).

Commercial Framing, Vendor Status, and Code Inclusion (Chi-square Tests)

This section explores the strong categorical associations using 100% stacked bar charts. The first visualization illustrates the overwhelming presence of advertisements in vendor-authored tutorials compared to community or educational sources. The second visualization contrasts how tutorials that include code are statistically less likely to display advertisements.

Test 1 (Vendor vs. Advertisement): χ² = 73.805, p = 8.6 × 10−18. This indicates a massive, statistically significant association where advertisements are nearly ubiquitous in vendor tutorials.
Test 2 (Code Inclusion vs. Advertisement): χ² = 14.768, p = 0.000122. This significant association demonstrates that tutorials containing code blocks were less likely to display commercial framing.

Methodology

We identified web-application security tutorials through web search, selecting the tutorials that appeared most frequently and ranked highest across multiple queries. We then characterized the tutorials along several dimensions using a structured coding scheme.
  1. Data Collection — We used the DuckDuckGo Search API to issue 21 queries capturing realistic developer search intents. For each query, we retrieved the first 50 results (1,050 total entries).
  2. Filtering & Selection — After removing duplicates, we had 872 distinct URLs. We computed visibility scores by summing rankings across all queries and selected the top 200 most visible tutorials.
  3. Tutorial Validation — We manually excluded videos, courses, PDFs, marketing pages, and non-instructional content. This left us with 132 tutorials in the final dataset.
  4. Data Analysis — We coded each tutorial for website type, content organization, code examples, marketing presence, and security standards references. Two authors independently coded 10 tutorials to establish reliability (κ ≈ 1.00).