Services

Dynamic Web Content Rendering & Caching

A reference implementation for transforming JavaScript-rendered web pages into complete HTML for search engines, crawlers and chatbots.

Reference Environment: Adobe Experience Manager 6.4 Core Technology: Playwright Browser-based Rendering Content Delivery: HTML Repository Apache Routing / Cache Operational Visibility: Rendering & Cached Content Dashboard

1. Executive Summary

In a previous implementation, a client was running AEM 6.4 pages where many pages contained sections whose content was rendered dynamically through JavaScript. The client needed a mechanism that could provide the resulting full page content to downstream consumers such as search engines, crawlers and chatbots.

A browser-based dynamic rendering approach was implemented. Following publication, the published page could be processed in the background, JavaScript executed in a real browser environment, and the resulting HTML captured and stored in a repository. Apache HTTP Server was used as part of the request-routing and caching approach, while a dashboard provided visibility into processing and the current cached-content repository.

Core pattern: AEM page → dynamic rendering → Playwright/browser execution → final HTML → repository/cache → Apache request routing → downstream consumers.

2. Client Environment

  • CMS: Adobe Experience Manager (AEM) 6.4
  • Page behavior: Many pages used JavaScript to render dynamic sections.
  • Need: Make the final rendered page content available as complete HTML for search engines, crawlers and chatbots.
  • Operational requirement: Generated HTML needed to be refreshed when published content changed.

3. Business / Technical Challenge

The page source available before client-side execution did not necessarily contain all of the content required by downstream consumers. Important sections were populated after JavaScript execution.

The implementation therefore needed to address three related concerns:

  1. Render the published page in a browser-capable environment.
  2. Capture the final HTML after dynamic sections had been populated.
  3. Serve or route requests to the generated HTML without requiring every downstream consumer to execute the page's JavaScript.

4. Solution Approach

The solution introduced a dynamic rendering layer between the published web experience and selected downstream requests. The renderer opened the published page in a browser environment, allowed the page's JavaScript to execute, and captured the resulting HTML.

AEM 6.4
Published
Page
Dynamic
Renderer
Playwright
Browser
Final
HTML
HTML
Repository
Apache
Routing /
Cache

4.1 Background HTML Generation

HTML generation was performed as a background process rather than requiring a browser-rendering operation for every consumer request. When a page was published, the final page could be generated using Playwright and stored in the repository.

4.2 Repository and Refresh

The generated HTML was retained in a repository that represented the current cached version of the page. The same repository was used when generated HTML needed to be refreshed or updated.

4.3 Apache Request Routing

Apache rules were used to identify relevant search-engine/crawler requests and route them through the dynamic-rendering/caching path. This allowed the generated HTML to be used for the applicable requests while the normal web experience remained available for other traffic.

4.4 Operational Dashboard

A dashboard was created to provide visibility into current processing and the current cached-content repository. This gave the implementation an operational view rather than treating generated HTML as an unmanaged collection of files.

5. Reference Architecture - Previous Implementation

The following architecture represents the previous implementation described in this case study. It is intentionally presented as a reference architecture, not as the current RenderXP product architecture.

1. Content publication

A page is published from AEM.

2. Background rendering

The published URL is processed by the dynamic rendering mechanism.

3. Browser execution

Playwright loads the page and executes its JavaScript.

4. HTML capture

The final rendered HTML is captured.

5. Repository storage

The generated HTML is stored in the repository.

6. Request routing / caching

Apache rules route applicable requests to the generated content.

7. Downstream consumption

Search engines, crawlers and chatbots can consume the complete HTML representation.

6. End-to-End Workflow

  1. AEM publishes or updates a page.
  2. The page enters the background rendering flow.
  3. The renderer opens the published URL using Playwright.
  4. JavaScript executes and dynamic sections are populated.
  5. The resulting page HTML is captured.
  6. The HTML is stored or refreshed in the repository.
  7. Apache routing/caching makes the generated representation available for applicable requests.
  8. The dashboard exposes processing and repository status.

7. Key Architectural Principles Demonstrated

  • Browser-based rendering: JavaScript-heavy pages can be rendered in an environment capable of executing the same browser-side logic used by the page.
  • Pre-generation: Rendering can occur before downstream consumption instead of adding browser-rendering work to every request.
  • Repository-backed content: Generated HTML can be retained and refreshed as content changes.
  • Selective routing: Apache can participate in determining which requests use the generated representation.
  • Operational visibility: A dashboard can expose rendering activity and cached-content state.

8. Relevance to RenderXP

This reference implementation provides the technical foundation for the RenderXP product direction. RenderXP productizes the same core problem-turning dynamic web pages into complete, reusable HTML snapshots-while introducing a more generalized platform model that is not tied exclusively to AEM.

Reference experience Product direction
AEM-specific dynamic renderer standalone rendering platform
Manual/implementation-specific processing persistent render jobs and scheduling
HTML repository managed snapshot repository
Apache routing independent snapshot serving / cache layer
Operational dashboard RenderXP dashboard and monitoring
Publish-driven refresh event-driven and scheduled refresh mechanisms