Jekyll2024-02-26T19:35:05+00:00https://engineering.rei.com/feed.xmlREI Co-op EngineeringWe're into open source and the outdoors. And in the spirit of giving back to the community (that's you), REI Co-op's engineers are sharing awesome ideas, thoughts, guides and other works related to our open source software.
XCUITest Automation: Page Components for iOS Test Automation2024-01-29T00:00:00+00:002024-01-29T00:00:00+00:00https://engineering.rei.com/mobile/xcuitest-page-components<h2 id="creating-stable-maintainable-user-interface-test-automation-in-swift">Creating Stable, Maintainable User Interface Test Automation in Swift</h2>
<p>When creating test automation to validate the performance and behavior of a complex software application, adhering to solid engineering practices will produce stable, maintainable tests. This is especially true when driving the application through its user interface. Page Object Model (POM) design patterns provide a solid framework to achieve this objective.</p>
<h3 id="functional-grouping-with-page-components">Functional grouping with page components</h3>
<p>In the <a href="xcuitest-page-object-models.html">previous article</a>, we showed how POM design patterns can be applied to implementing automated interactions with iOS applications through the XCUITest framework. Modeling each view with a single comprehensive page class is adequate for simple views, but factoring the features of more complex view into functional grouping can simplify interactions with these features.</p>
<p>Modeling an application based solely on its views produces a very flat model. It’s quite common for a view to contain groups of elements that are logically associated (e.g. - billing address on an order information view). It’s also common to encounter views with multiple occurrences of an element grouping (e.g. - item tiles on a search results view). Factoring these grouping out into <strong>page components</strong> can greatly enrich your models, presenting a conceptual framework that automation developers will recognize.</p>
<p>In the following example, the title block elements of the product details view are modeled in a page component:</p>
<h5 id="productdetailsview-excerpt">ProductDetailsView excerpt</h5>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//</span>
<span class="c1">// ProductView.swift</span>
<span class="c1">// Copyright © 2024 REI. All rights reserved.</span>
<span class="c1">//</span>
<span class="kd">import</span> <span class="kt">XCTest</span>
<span class="kd">class</span> <span class="kt">ProductDetailsView</span><span class="o"><</span><span class="kt">T</span><span class="p">:</span> <span class="kt">BaseView</span><span class="o">></span><span class="p">:</span> <span class="kt">NavigationHub</span> <span class="p">{</span>
<span class="kd">lazy</span> <span class="k">var</span> <span class="nv">titleBlock</span><span class="p">:</span> <span class="kt">ProductTitleBlock</span> <span class="o">=</span> <span class="p">{</span>
<span class="k">return</span> <span class="kt">ProductTitleBlock</span><span class="p">(</span><span class="k">self</span><span class="p">)</span>
<span class="p">}()</span>
<span class="c1">// -._.-'¯'-._.-'¯'-._.-'¯'-._.-'¯'-._.-'¯'-._.-'¯'-._.-'¯'-</span>
<span class="p">}</span>
</code></pre></div></div>
<h5 id="producttitleblock">ProductTitleBlock</h5>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//</span>
<span class="c1">// ProductTitleBlock.swift</span>
<span class="c1">// Copyright © 2024 REI. All rights reserved.</span>
<span class="c1">//</span>
<span class="kd">import</span> <span class="kt">XCTest</span>
<span class="kd">class</span> <span class="kt">ProductTitleBlock</span><span class="o"><</span><span class="kt">T</span><span class="p">:</span> <span class="kt">BaseView</span><span class="o">></span> <span class="p">{</span>
<span class="kd">enum</span> <span class="kt">Element</span> <span class="p">{</span>
<span class="k">case</span> <span class="n">product_brand</span>
<span class="k">case</span> <span class="n">product_title</span>
<span class="k">case</span> <span class="n">style_or_sku</span>
<span class="kd">func</span> <span class="nf">locator</span><span class="p">(</span><span class="n">_</span> <span class="nv">context</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="o">-></span> <span class="kt">XCUIElement</span> <span class="p">{</span>
<span class="k">switch</span> <span class="k">self</span> <span class="p">{</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">product_brand</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">staticTexts</span><span class="p">[</span><span class="s">"TitleBlock.productBrand.label"</span><span class="p">]</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">product_title</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">staticTexts</span><span class="p">[</span><span class="s">"TitleBlock.productName.label"</span><span class="p">]</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">style_or_sku</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">staticTexts</span><span class="p">[</span><span class="s">"TitleBlock.styleOrSku.label"</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">private</span> <span class="k">let</span> <span class="nv">parent</span><span class="p">:</span> <span class="kt">T</span>
<span class="nf">init</span><span class="p">(</span><span class="n">_</span> <span class="nv">parent</span><span class="p">:</span> <span class="kt">T</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">parent</span> <span class="o">=</span> <span class="n">parent</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">getProductId</span><span class="p">()</span> <span class="o">-></span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">return</span> <span class="kt">Element</span><span class="o">.</span><span class="n">style_or_sku</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">parent</span><span class="o">.</span><span class="kt">APP</span><span class="p">)</span><span class="o">.</span><span class="n">label</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>In the preceding code, note how the <strong>ProductTitleBlock</strong> page component is defined as a lazy-initialize property of the main <strong>ProductDetailsPage</strong> class. The page component itself declares a locator enumeration that defined three title block elements, with the <code class="highlighter-rouge">getProductId()</code> function providing access to the corresponding content.</p>
<p>Note that the context used to locate the product ID element is derived from the component’s parent (the product details view). In this case, the search context for component elements is the target application, because there’s no specific container that groups the title block elements. However, it’s common for components to represent specific sub-contexts - a scroll view for a multi-item collection or a cell that contains one of these items. More on search contexts and component collections later.</p>
<p>The test class shown below demonstrates an how an automated test might use this page model to verify that the expected product ID is shown in the title block of the product details view.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="kt">MSAProductTabTests</span><span class="p">:</span> <span class="kt">XCTestCase</span> <span class="p">{</span>
<span class="kd">func</span> <span class="nf">testSKUShownOnPDP_NoColor_NoSize</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">sku</span> <span class="o">=</span> <span class="s">"510-145-0012"</span>
<span class="k">guard</span> <span class="kc">nil</span> <span class="o">!=</span> <span class="n">ascentHomeView</span><span class="o">.</span><span class="nf">launchTheApplication</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">productDetailsView</span> <span class="o">=</span> <span class="n">ascentHomeView</span><span class="o">.</span><span class="n">searchBar</span><span class="o">.</span><span class="nf">searchForSingleItem</span><span class="p">(</span><span class="n">sku</span><span class="p">)</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="kt">XCTAssertEqual</span><span class="p">(</span><span class="n">productDetailsView</span><span class="o">.</span><span class="n">titleBlock</span><span class="o">.</span><span class="nf">getProductId</span><span class="p">(),</span> <span class="n">sku</span><span class="p">,</span> <span class="s">"Did not find product SKU: </span><span class="se">\(</span><span class="n">sku</span><span class="se">)</span><span class="s">"</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>As shown above, access to the product ID is provided by the <strong>ProductTitleBlock</strong> page component via the <em>[titleBlock]</em> property of the <strong>ProductDetailsView</strong> object. The logical grouping provided by this structure helps to differentiate the features of this group relative to other components of the view. This can be extremely helpful when implementing tests that interact with complex views comprised of dozens of elements. Each component provides focused access to a small set of logically associated elements, while a flat model would inundate the test author with an unstructured collection of every element of the entire view.</p>
<h3 id="page-component-with-unique-search-context">Page component with unique search context</h3>
<p>The <strong>ProductTitleBlock</strong> page component model shown above illustrates a <em>“virtual component”</em> - a page component with no associated container element. These sorts of components are great for organizing and logically segmenting the functionality of complex views. The presence of a unique container element that encapsulated the features of a page component can increase the cohesion, safety, and efficiency of your model by providing a limited scope for element selection - the <strong>search context</strong>.</p>
<p>The following example demonstrates how to define and use a pre component search context:</p>
<h4 id="defining-search-context-for-singleton-page-component">Defining search context for singleton page component</h4>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//</span>
<span class="c1">// ProductInventoryBlockSubView.swift</span>
<span class="c1">// MSAMobileApp-UITests</span>
<span class="c1">//</span>
<span class="c1">// Created by John Comstock on 8/2/21.</span>
<span class="c1">// Copyright © 2021 REI. All rights reserved.</span>
<span class="c1">//</span>
<span class="kd">import</span> <span class="kt">XCTest</span>
<span class="kd">class</span> <span class="kt">ProductAvailabilityHeader</span><span class="o"><</span><span class="kt">T</span><span class="p">:</span> <span class="kt">NavigationHub</span><span class="o">></span> <span class="p">{</span>
<span class="kd">enum</span> <span class="kt">Element</span> <span class="p">{</span>
<span class="k">case</span> <span class="n">view_context</span>
<span class="k">case</span> <span class="n">onhand_value</span>
<span class="kd">func</span> <span class="nf">locator</span><span class="p">(</span><span class="n">_</span> <span class="nv">context</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="o">-></span> <span class="kt">XCUIElement</span> <span class="p">{</span>
<span class="k">switch</span> <span class="k">self</span> <span class="p">{</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">view_context</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">otherElements</span><span class="p">[</span><span class="s">"AvailabilityHeader.container.element"</span><span class="p">]</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">onhand_value</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">staticTexts</span><span class="p">[</span><span class="s">"OnHand.availability.value"</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">private</span> <span class="k">let</span> <span class="nv">parent</span><span class="p">:</span> <span class="kt">T</span>
<span class="kd">private</span> <span class="k">let</span> <span class="nv">context</span><span class="p">:</span> <span class="kt">XCUIElement</span>
<span class="nf">init</span><span class="p">(</span><span class="n">_</span> <span class="nv">parent</span><span class="p">:</span> <span class="kt">T</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">parent</span> <span class="o">=</span> <span class="n">parent</span>
<span class="k">self</span><span class="o">.</span><span class="n">context</span> <span class="o">=</span> <span class="kt">Element</span><span class="o">.</span><span class="n">view_context</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">parent</span><span class="o">.</span><span class="kt">APP</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">getNearbyQuantity</span><span class="p">()</span> <span class="o">-></span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">return</span> <span class="kt">Element</span><span class="o">.</span><span class="n">nearby_value</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">context</span><span class="p">)</span><span class="o">.</span><span class="n">label</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Note that the initializer receives a reference to the page or component that’s including the <strong>ProductAvailabilityHeader</strong> component - the <em>parent</em>. The initializer derives the component’s search context from the parent, and this becomes the context for element searches within the component itself. For example, the <code class="highlighter-rouge">getNearbyQuantity()</code> function derives the locator to the nearby store quantity element relative to this context.</p>
<p>It’s not uncommon for page components to be initialized with a pre-assembled search context. However, this distributes the responsibility for defining and handling the attributes and behaviors of the component to two different classes - the page component and its parent. Prefer confining all component-specific operations to the page component class itself. This increases cohesion and simplifies ongoing maintenance of the page model implementation.</p>
<h3 id="search-context-for-page-component-collection-entries">Search context for page component collection entries</h3>
<p>Search contexts aren’t required to be unique. It’s commonplace to encounter multiple inherently indistinguishable container elements that encapsulate functionally equivalent components (e.g. - search result items). When modeling these sorts of components, definitive identification of each component must typically derive from attributes of elements within the component. Ideally, however, each container element will provided a unique identifier. In the following example, the style identifier for the product presented in each search result component is stored in the <em>[value]</em> property of the containing <strong>cell</strong> element:</p>
<h5 id="defining-unique-search-contexts-for-page-component-collection-entries">Defining unique search contexts for page component collection entries</h5>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//</span>
<span class="c1">// SeaarchResult.swift</span>
<span class="c1">// Copyright © 2024 REI. All rights reserved.</span>
<span class="c1">//</span>
<span class="kd">import</span> <span class="kt">XCTest</span>
<span class="kd">class</span> <span class="kt">SearchResult</span><span class="o"><</span><span class="kt">T</span><span class="p">:</span> <span class="kt">BaseView</span><span class="o">></span> <span class="p">{</span>
<span class="kd">enum</span> <span class="kt">Element</span> <span class="p">{</span>
<span class="k">case</span> <span class="nf">result_cell</span><span class="p">(</span><span class="kt">String</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span><span class="p">)</span>
<span class="k">case</span> <span class="n">product_brand</span>
<span class="kd">func</span> <span class="nf">locator</span><span class="p">(</span><span class="n">_</span> <span class="nv">context</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="o">-></span> <span class="kt">XCUIElement</span> <span class="p">{</span>
<span class="k">switch</span> <span class="k">self</span> <span class="p">{</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">result_cell</span><span class="p">(</span><span class="k">let</span> <span class="nv">id</span><span class="p">):</span>
<span class="k">if</span> <span class="k">let</span> <span class="nv">styleId</span> <span class="o">=</span> <span class="n">id</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">predicate</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"identifier == %@ AND value == %@"</span><span class="p">,</span> <span class="kt">SearchResultConstants</span><span class="o">.</span><span class="kt">RESULT_CELL</span><span class="p">,</span> <span class="n">styleId</span><span class="p">)</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">cells</span><span class="o">.</span><span class="nf">matching</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span><span class="o">.</span><span class="n">element</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">cells</span><span class="p">[</span><span class="kt">SearchResultConstants</span><span class="o">.</span><span class="kt">RESULT_CELL</span><span class="p">]</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">product_brand</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">staticTexts</span><span class="p">[</span><span class="s">"SearchResultCell.productBrandLabel"</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">private</span> <span class="k">let</span> <span class="nv">parent</span><span class="p">:</span> <span class="kt">T</span>
<span class="kd">private</span> <span class="k">let</span> <span class="nv">styleId</span><span class="p">:</span> <span class="kt">String</span>
<span class="kd">private</span> <span class="k">let</span> <span class="nv">context</span><span class="p">:</span> <span class="kt">XCUIElement</span>
<span class="nf">init</span><span class="p">?(</span><span class="n">_</span> <span class="nv">context</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">,</span> <span class="k">in</span> <span class="nv">parent</span><span class="p">:</span> <span class="kt">T</span><span class="p">)</span> <span class="p">{</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">styleId</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">value</span> <span class="k">as?</span> <span class="kt">String</span> <span class="k">else</span> <span class="p">{</span>
<span class="kt">XCTFail</span><span class="p">(</span><span class="s">"Failed extracting style ID from search result cell value"</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">nil</span>
<span class="p">}</span>
<span class="k">self</span><span class="o">.</span><span class="n">parent</span> <span class="o">=</span> <span class="n">parent</span>
<span class="k">self</span><span class="o">.</span><span class="n">styleId</span> <span class="o">=</span> <span class="n">styleId</span>
<span class="k">self</span><span class="o">.</span><span class="n">context</span> <span class="o">=</span> <span class="kt">Element</span><span class="o">.</span><span class="nf">result_cell</span><span class="p">(</span><span class="n">styleId</span><span class="p">)</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">parent</span><span class="o">.</span><span class="kt">APP</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">getProductBrand</span><span class="p">()</span> <span class="o">-></span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">return</span> <span class="kt">Element</span><span class="o">.</span><span class="n">product_brand</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">context</span><span class="p">)</span><span class="o">.</span><span class="n">label</span>
<span class="p">}</span>
<span class="kd">static</span> <span class="kd">func</span> <span class="nf">buildResultsList</span><span class="p">(</span><span class="n">_</span> <span class="nv">parent</span><span class="p">:</span> <span class="kt">T</span><span class="p">)</span> <span class="o">-></span> <span class="p">[</span><span class="kt">SearchResult</span><span class="o"><</span><span class="kt">T</span><span class="o">></span><span class="p">]?</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">selector</span> <span class="o">=</span> <span class="kt">Element</span><span class="o">.</span><span class="nf">result_cell</span><span class="p">()</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">parent</span><span class="o">.</span><span class="kt">APP</span><span class="p">)</span>
<span class="k">guard</span> <span class="n">selector</span><span class="o">.</span><span class="nf">waitForExistence</span><span class="p">(</span><span class="nv">timeout</span><span class="p">:</span> <span class="kt">Constants</span><span class="o">.</span><span class="n">standardTimeout</span><span class="p">)</span> <span class="k">else</span> <span class="p">{</span>
<span class="kt">XCTFail</span><span class="p">(</span><span class="s">"Failed to load Search Results"</span><span class="p">)</span>
<span class="k">return</span> <span class="kc">nil</span>
<span class="p">}</span>
<span class="k">var</span> <span class="nv">searchResults</span><span class="p">:</span> <span class="p">[</span><span class="kt">SearchResult</span><span class="o"><</span><span class="kt">T</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">element</span> <span class="k">in</span> <span class="n">selector</span><span class="o">.</span><span class="n">allElementsBoundByAccessibilityElement</span> <span class="p">{</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">searchResult</span> <span class="o">=</span> <span class="kt">SearchResult</span><span class="p">(</span><span class="n">element</span><span class="p">,</span> <span class="nv">in</span><span class="p">:</span> <span class="n">parent</span><span class="p">)</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="kc">nil</span>
<span class="p">}</span>
<span class="n">searchResults</span><span class="o">.</span><span class="nf">append</span><span class="p">(</span><span class="n">searchResult</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">searchResults</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">struct</span> <span class="kt">SearchResultConstants</span> <span class="p">{</span>
<span class="kd">static</span> <span class="k">let</span> <span class="nv">RESULT_CELL</span> <span class="o">=</span> <span class="s">"SearchResult.cell"</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The <strong>SearchResult</strong> class provides a static function that builds a collection of search result page components. Note how the <code class="highlighter-rouge">buildResultsList()</code> function employs the no-argument form of the <strong>result_cell</strong> constant to acquire a list of every search result container element in the view. The implementation iterates over the selected container elements via their accessibility identifiers, which are somewhat more durable than the references produced by <code class="highlighter-rouge">allElementsBoundByIndex</code>. However, these references are still at risk of going stale when the view gets repopulated after the search results are scrolled or filters are applied.</p>
<p>To avoid subsequent reference mismatch issues, the <strong>SearchResult</strong> initializer produces a fully qualified reference by specifying the style ID as the argument for the <strong>result_cell</strong> element locator constant. This ensures that we always select the search result we expect. If the specified element exists, the associated search result item is guaranteed to show the expected product. If the specified element isn’t found later (e.g. - excluded by an applied filter), we can be certain that the corresponding search result item is actually gone. Note that the initializer is specified as optional; if the style ID can’t be acquired with the specified element reference, the initializer returns <code class="highlighter-rouge">nil</code>. (This is academic, though, since the initializer also registers a test failure.)</p>
<h2 id="summary">Summary</h2>
<p>Page object design patterns provided many benefits when applied to user interface automation of iOS applications. Modeling related groups of elements as page components can make your designs even more cohesive, comprehensible, and flexible. The logical encapsulation provided by page components can reduce the complexity of implementing and interacting with the associated application features. This is especially true for views that contain multiple instances of the same component.</p>
<blockquote>
<p>Written with <a href="https://stackedit.io/">StackEdit</a>.</p>
</blockquote>Scott Babcockscbabco@rei.comCreating Stable, Maintainable User Interface Test Automation in SwiftXCUITest Automation: Page Object Models for iOS Test Automation2024-01-26T00:00:00+00:002024-01-26T00:00:00+00:00https://engineering.rei.com/mobile/xcuitest-page-object-models<h2 id="creating-stable-maintainable-user-interface-test-automation-in-swift">Creating Stable, Maintainable User Interface Test Automation in Swift</h2>
<p>When creating test automation to validate the performance and behavior of a complex software application, adhering to solid engineering practices will produce stable, maintainable tests. This is especially true when driving the application through its user interface. Page Object Model (POM) design patterns provide a solid framework to achieve this objective.</p>
<h3 id="implementing-page-object-model-pom-patterns-in-swift">Implementing Page Object Model (POM) patterns in Swift</h3>
<p>Page Object Model (POM) design patterns, originally conceived to automate web application user interface tests, can be applied to testing iOS application through the XCUITest framework. Following a page-model approach produces structured interfaces that your tests can use to drive the target application and verify expected behaviors.</p>
<ul>
<li>Each application main view is associated with a separate page model class.</li>
<li>Functions that trigger navigation wait for the expected landing view to appear and return an instance of the corresponding page model class.</li>
<li>To facilitate navigation rewinding, stackable page model classes are parameterized, with the type parameter supplying the return type for back/close operations.</li>
<li>Groups of elements that comprise distinct units of application functionality can be modeled as <a href="xcuitest-page-components.html">page component</a> classes.</li>
<li>Repeated element groupings (e.g. - search result items) should be modeled as <a href="xcuitest-page-components.html#search-context-for-page-component-collection-entries">page component collections</a>.</li>
<li><a href="xcuitest-element-locator-enumerations.html">Define element locators in Swift enumerations</a> to avoid duplication of element types and attributes.</li>
</ul>
<h3 id="synchronization---the-lynchpin-of-stable-automation">Synchronization - The lynchpin of stable automation</h3>
<p>In user interface automation, everything begins and ends with definite synchronization between the application under test (AUT) and the automation that drives it. This isn’t just a figure of speech; it’s literally true.</p>
<ul>
<li>Before interacting with a view that’s just been opened, perform load-completion checks to ensure that the application is ready for action.</li>
<li>After performing an action that triggers an application state change (expanding a dropdown, opening a modal, etc.), wait for the expected response.</li>
</ul>
<p>Adhering to these rules ensures that your automation interacts smoothly and reliably with the application. It also enables you to recognize unexpected behavior at the point where it’s encountered, failing with clear diagnostic output indicating what went wrong. Remember…</p>
<blockquote>
<p>RULE #1: Fail Fast with Detailed Diagnostics</p>
</blockquote>
<h4 id="the-synchronizedview-protocol">The SynchronizedView protocol</h4>
<p>In the REI automation framework, all of our application view page models implement a <strong>SynchronizedView</strong> protocol. The methods that facilitate page load synchronization are:</p>
<ul>
<li><strong><code class="highlighter-rouge">waitForView(required: Bool) -> Self?</code></strong>
Wait for the view associated with this model to finish loading.</li>
<li><strong><code class="highlighter-rouge">checkViewCriteria() -> String?</code></strong>
Determine if all evaluated criteria indicate that the view associated with this model has finished loading.</li>
<li><strong><code class="highlighter-rouge">hasNetworkSpinner() -> Bool</code></strong>
Determine if transition to the view associated with this model initially displays a network activity spinner.</li>
<li><strong><code class="highlighter-rouge">launchTheApplication() -> Self?</code></strong>
Launch the target application and wait for the expected view to load.</li>
</ul>
<p>The foundation of the synchronization provided by the framework is the <strong><code class="highlighter-rouge">checkViewCriteria()</code></strong> function. Each page class declares a view-specific implementation of this function. As described in the function header, the implementation returns <code class="highlighter-rouge">nil</code> when all evaluated criteria are met. If any evaluated criterion is unmet, the function returns a string that describes the issue. If after twenty seconds the function doesn’t return <code class="highlighter-rouge">nil</code>, the affected test is terminated with the description of the unmet criterion.</p>
<blockquote>
<p><strong>NOTE</strong>: The <code class="highlighter-rouge">checkViewCriteria()</code> function should perform instantaneous evaluations; it should check for each defined criterion and return immediately upon encountering one that’s not met. The implementation of <code class="highlighter-rouge">waitForView()</code> polls this function repeatedly until it returns <code class="highlighter-rouge">nil</code> or the maximum delay interval has elapsed.</p>
</blockquote>
<h5 id="page-object-model-protocol-part-1---synchronization">Page Object Model Protocol (Part 1 - Synchronization)</h5>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//</span>
<span class="c1">// SynchronizedView.swift (Part 1)</span>
<span class="c1">// Copyright © 2024 REI. All rights reserved.</span>
<span class="c1">//</span>
<span class="kd">import</span> <span class="kt">XCTest</span>
<span class="kd">protocol</span> <span class="kt">SynchronizedView</span> <span class="p">{</span>
<span class="c1">/// Determine if transition to the view associated with this model initially displays a network activity spinner.</span>
<span class="c1">/// > Note: The default implementation of this function returns `false`; declare an override returning `true` if the associated view opens with a spinner.</span>
<span class="c1">/// - Returns: `true` if expecting an initial network spinner</span>
<span class="kd">func</span> <span class="nf">hasNetworkSpinner</span><span class="p">()</span> <span class="o">-></span> <span class="kt">Bool</span>
<span class="c1">/// Determine if all evaluated criteria indicate that the view associated with this model has finished loading.</span>
<span class="c1">/// - Returns: `nil` if all evaluated criteria are satisfied; otherwise, description of unsatisfied view load criterion</span>
<span class="kd">func</span> <span class="nf">checkViewCriteria</span><span class="p">()</span> <span class="o">-></span> <span class="kt">String</span><span class="p">?</span>
<span class="c1">/// Launch the target application and wait for the expected view to load.</span>
<span class="c1">/// - Returns: this view model; `nil` if expected view doesn't load</span>
<span class="kd">func</span> <span class="nf">launchTheApplication</span><span class="p">()</span> <span class="o">-></span> <span class="k">Self</span><span class="p">?</span>
<span class="c1">/// Terminate the target application.</span>
<span class="kd">func</span> <span class="nf">terminateTheApplication</span><span class="p">()</span>
<span class="c1">/// Wait for the view associated with this model to finish loading.</span>
<span class="c1">/// > Note: If ``hasNetworkSpinner()`` returns `true`, this function will wait for the network spinner to vanish prior to evaluation of view load criteria.</span>
<span class="c1">/// > Note: If [required] is `true` and the standard load interval expires, the current test will fail with a message indicating this model and the description of the unsatified view load criterion.</span>
<span class="c1">/// - Parameter required: [optional] `false` if incomplete loading of the associated view is allowed (default = `true`)</span>
<span class="c1">/// - Returns: this view model; `nil` if view transition fails</span>
<span class="kd">func</span> <span class="nf">waitForView</span><span class="p">(</span><span class="nv">required</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">)</span> <span class="o">-></span> <span class="k">Self</span><span class="p">?</span>
<span class="p">}</span>
<span class="kd">extension</span> <span class="kt">SynchronizedView</span> <span class="p">{</span>
<span class="kd">func</span> <span class="nf">waitForView</span><span class="p">(</span><span class="nv">required</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="kc">true</span><span class="p">)</span> <span class="o">-></span> <span class="k">Self</span><span class="p">?</span> <span class="p">{</span>
<span class="k">if</span> <span class="nf">hasNetworkSpinner</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// TODO: application-dependent spinner check</span>
<span class="p">}</span>
<span class="k">if</span> <span class="k">let</span> <span class="nv">result</span> <span class="o">=</span> <span class="kt">Utility</span><span class="o">.</span><span class="nf">waitForExpression</span><span class="p">(</span><span class="nf">checkViewCriteria</span><span class="p">(),</span> <span class="nv">timeout</span><span class="p">:</span> <span class="kt">Constants</span><span class="o">.</span><span class="n">standardTimeout</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="kd">required</span> <span class="p">{</span> <span class="kt">Utility</span><span class="o">.</span><span class="nf">failWithAlertCheck</span><span class="p">(</span><span class="s">"Failed opening '</span><span class="se">\(</span><span class="nf">getViewName</span><span class="p">()</span><span class="se">)</span><span class="s">' view: </span><span class="se">\(</span><span class="n">result</span><span class="se">)</span><span class="s">"</span><span class="p">)</span> <span class="p">}</span>
<span class="k">return</span> <span class="kc">nil</span>
<span class="p">}</span>
<span class="k">return</span> <span class="k">self</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="navigation---building-a-chain-of-synchronized-views">Navigation - Building a chain of synchronized views</h3>
<p>The navigation strategy employed by the REI automation framework explicitly defines the relationships between associated views:</p>
<ul>
<li>Every method that triggers a view transition returns an instance of the page class for the expected landing view.</li>
<li>To facilitate retracing steps in the navigation stack, each page object retains the instance that spawned it (the origin).</li>
<li>Each navigation stack originates from a root view, which has no origin.</li>
<li>Non-root page classes are parameterized, and the parameter type specified when each instance is created resolves the class of the origin when stepping back down the stack.</li>
</ul>
<p>The first point of this strategy is the key to effortless stability at view transitions. When your model returns a new page object, the framework has already ensured that the corresponding view is present and prepared for the next action. This is accomplished in <code class="highlighter-rouge">openView()</code> and <code class="highlighter-rouge">returnToOrigin()</code> via the <code class="highlighter-rouge">waitForView()</code> function, which uses the <code class="highlighter-rouge">checkViewCriteria()</code> function of the new page object for synchronization.</p>
<h5 id="page-object-model-protocol-part-2---object-chaining">Page Object Model Protocol (Part 2 - Object Chaining)</h5>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//</span>
<span class="c1">// SynchronizedView.swift (Part 2)</span>
<span class="c1">// Copyright © 2023 REI. All rights reserved.</span>
<span class="c1">//</span>
<span class="kd">import</span> <span class="kt">XCTest</span>
<span class="kd">protocol</span> <span class="kt">SynchronizedView</span> <span class="p">{</span>
<span class="c1">/// Origin of this view.</span>
<span class="c1">/// > Note: The origin is the view that the application returns to when the user taps **Back** or **Close**; may be `nil`</span>
<span class="k">var</span> <span class="nv">origin</span><span class="p">:</span> <span class="kt">BaseView</span><span class="p">?</span> <span class="p">{</span> <span class="k">get</span> <span class="p">}</span>
<span class="c1">/// Target application for this view.</span>
<span class="k">var</span> <span class="nv">APP</span><span class="p">:</span> <span class="kt">XCUIApplication</span> <span class="p">{</span> <span class="k">get</span> <span class="p">}</span>
<span class="c1">/// Required constructor:</span>
<span class="c1">/// - Parameters</span>
<span class="c1">/// - origin: [optional] Origin of this view (default = `nil`)</span>
<span class="c1">/// - app: Target application for this instance</span>
<span class="c1">/// - See: ``origin``</span>
<span class="nf">init</span><span class="p">(</span><span class="n">_</span> <span class="nv">origin</span><span class="p">:</span> <span class="kt">BaseView</span><span class="p">?,</span> <span class="nv">app</span><span class="p">:</span> <span class="kt">XCUIApplication</span><span class="p">)</span>
<span class="c1">/// Open the indicated view by tapping the specified element.</span>
<span class="c1">/// > Note: If [required] is `true` and the standard load interval expires, the current test will fail with a message indicating this model and the description of the unsatified view load criterion.</span>
<span class="c1">/// - Parameters:</span>
<span class="c1">/// - viewType: Class of the automation model associated with the view that opens after tapping the specified element</span>
<span class="c1">/// - targetApp: [optional] Target application for view being opened (default = `self.APP`)</span>
<span class="c1">/// - element: Element that triggers the expected view transition</span>
<span class="c1">/// - fixed: [optional] `true` if the specified element is non-scrollable (default = `false`)</span>
<span class="c1">/// - required: [optional] `false` if incomplete loading of the associated view is allowed (default = `true`)</span>
<span class="c1">/// - Returns: target view model; `nil` if view transition fails</span>
<span class="c1">/// - See: ``waitForView``</span>
<span class="kd">func</span> <span class="n">openView</span><span class="o"><</span><span class="kt">T</span><span class="p">:</span> <span class="kt">BaseView</span><span class="o">></span><span class="p">(</span><span class="n">_</span> <span class="nv">viewType</span><span class="p">:</span> <span class="kt">T</span><span class="o">.</span><span class="k">Type</span><span class="p">,</span> <span class="nv">targetApp</span><span class="p">:</span> <span class="kt">XCUIApplication</span><span class="p">?,</span> <span class="n">byTapping</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">,</span>
<span class="nv">fixed</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">,</span> <span class="nv">required</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">)</span> <span class="o">-></span> <span class="kt">T</span><span class="p">?</span>
<span class="c1">/// Tap **Back** / **Close** to navigate back to the origin of the current view, as registered in this model.</span>
<span class="c1">/// > Note: This function is unsupported for "navogation hub" models, as these represent the bottom of the nav stack and therefore have no related **Back** action.</span>
<span class="c1">/// - Parameters:</span>
<span class="c1">/// - element: [optional] Element that triggers the expected view transition (default = `nil`)</span>
<span class="c1">/// - fixed: [optional] `true` if the specified element is non-scrollable (default = `false`)</span>
<span class="c1">/// - required: [optional] `false` if incomplete loading of the associated view is allowed (default = `true`)</span>
<span class="c1">/// - Returns: origin view model; `nil` if view transition fails</span>
<span class="c1">/// - See: ``origin``</span>
<span class="kd">func</span> <span class="n">returnToOrigin</span><span class="o"><</span><span class="kt">T</span><span class="p">:</span> <span class="kt">BaseView</span><span class="o">></span><span class="p">(</span><span class="n">byTapping</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">?,</span> <span class="nv">fixed</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">,</span> <span class="nv">required</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">)</span> <span class="o">-></span> <span class="kt">T</span><span class="p">?</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="best-practices-for-stable-automation-models">Best practices for stable automation models</h3>
<p>The REI framework implements several patterns that promote stable, reliable operation of our user-interface test automation. The full benefits of these facilities rely on the proper implementation of protocol functions within the page classes.</p>
<ul>
<li>Use the <code class="highlighter-rouge">openView()</code> and <code class="highlighter-rouge">returnToOrigin()</code> functions for all view transitions:
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">func</span> <span class="nf">openKnowledgeBase</span><span class="p">()</span> <span class="o">-></span> <span class="kt">KnowledgeBaseHome</span><span class="o"><</span><span class="kt">SettingsHomeView</span><span class="o">></span><span class="p">?</span> <span class="p">{</span>
<span class="k">return</span> <span class="nf">openView</span><span class="p">(</span><span class="kt">KnowledgeBaseHome</span><span class="o"><</span><span class="kt">SettingsHomeView</span><span class="o">>.</span><span class="k">self</span><span class="p">,</span> <span class="nv">byTapping</span><span class="p">:</span> <span class="kt">Element</span><span class="o">.</span><span class="n">knowledge_base_link</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="kt">APP</span><span class="p">))</span>
<span class="p">}</span>
</code></pre></div> </div>
</li>
<li>Perform explicit synchronization after each action that triggers a state transition (e.g. - switch filter mode):
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">func</span> <span class="nf">switchMode</span><span class="p">(</span><span class="n">via</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">Element</span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">reference</span> <span class="o">=</span> <span class="n">element</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="kt">APP</span><span class="p">)</span>
<span class="k">if</span> <span class="o">!</span><span class="n">reference</span><span class="o">.</span><span class="n">isSelected</span> <span class="p">{</span>
<span class="n">reference</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="n">reference</span><span class="o">.</span><span class="nf">wait</span><span class="p">(</span><span class="nv">until</span><span class="p">:</span> <span class="p">\</span><span class="o">.</span><span class="n">isSelected</span><span class="p">)</span>
<span class="k">guard</span> <span class="n">reference</span><span class="o">.</span><span class="n">isSelected</span> <span class="k">else</span> <span class="p">{</span>
<span class="kt">XCTFail</span><span class="p">(</span><span class="s">"Filter mode failed to become selected"</span><span class="p">)</span>
<span class="k">return</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div> </div>
</li>
<li><strong>NOTE</strong>: Each of these examples ensures that the transition is complete and the application is in the expected state before returning to the caller. This practice, called <strong><em>exit-state synchronization</em></strong>, is far more efficient and reliable than adding synchronization at the beginning of down-stream functions.</li>
</ul>
<h3 id="best-practices-for-test-implementation">Best practices for test implementation</h3>
<p>Adhering to a few core practices affords your tests the greatest benefit from the synchronization and stability provided by the REI framework:</p>
<ul>
<li>Only the initial page object is instantiated directly by the tests, typically in the <code class="highlighter-rouge">setupWithError()</code> function.</li>
<li>All other page objects are returned by functions that trigger view transitions.</li>
<li>Use the <code class="highlighter-rouge">guard let</code> pattern to resolve the optional page object values returned by transition-triggering functions.</li>
</ul>
<p>The example test class shown below demonstrates these practices. Note that the test code doesn’t require explicit verification of landing views at transitions, because the model does this automatically.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">import</span> <span class="kt">XCTest</span>
<span class="kd">class</span> <span class="kt">MSABasicNavigationTests</span><span class="p">:</span> <span class="kt">XCTestCase</span> <span class="p">{</span>
<span class="kd">private</span> <span class="k">var</span> <span class="nv">ascentHomeView</span><span class="p">:</span> <span class="kt">AscentHomeView</span><span class="o">!</span>
<span class="k">override</span> <span class="kd">func</span> <span class="nf">setUpWithError</span><span class="p">()</span> <span class="k">throws</span> <span class="p">{</span>
<span class="n">ascentHomeView</span> <span class="o">=</span> <span class="kt">AscentHomeView</span><span class="p">(</span><span class="nv">app</span><span class="p">:</span> <span class="kt">XCUIApplication</span><span class="o">.</span><span class="n">testingApp</span><span class="p">)</span>
<span class="c1">// stop immediately on failure</span>
<span class="n">continueAfterFailure</span> <span class="o">=</span> <span class="kc">false</span>
<span class="p">}</span>
<span class="k">override</span> <span class="kd">func</span> <span class="nf">tearDownWithError</span><span class="p">()</span> <span class="k">throws</span> <span class="p">{</span>
<span class="n">ascentHomeView</span> <span class="o">=</span> <span class="kc">nil</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">testNavigationToAscentKnowledgeBase</span><span class="p">()</span> <span class="p">{</span>
<span class="k">guard</span> <span class="kc">nil</span> <span class="o">!=</span> <span class="n">ascentHomeView</span><span class="o">.</span><span class="nf">launchTheApplication</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">settingsHomeView</span> <span class="o">=</span> <span class="n">ascentHomeView</span><span class="o">.</span><span class="n">footerBar</span><span class="o">.</span><span class="nf">selectSettingsTab</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">ascentKnowledgeBase</span> <span class="o">=</span> <span class="n">settingsHomeView</span><span class="o">.</span><span class="nf">openKnowledgeBase</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="k">guard</span> <span class="kc">nil</span> <span class="o">!=</span> <span class="n">ascentKnowledgeBase</span><span class="o">.</span><span class="nf">backToOrigin</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The preceding example demonstrates the page object chaining behavior of the framework. The output of each function that triggers a view transition is the page object for the view that was activated by the function. If a <code class="highlighter-rouge">nil</code> is returned, the test exits immediately. Because we set <code class="highlighter-rouge">continueAfterFailure</code> to <code class="highlighter-rouge">false</code> during test setup, this is academic - an error has already been registered explaining the issue. However, it’s still a good practice.</p>
<h4 id="handling-transitions-with-interstitial-toast">Handling transitions with interstitial “toast”</h4>
<p>When an action triggers a view transition that also includes a “toast” message, special handling is required to capture and respond to the “toast” message and synchronize with the newly activated view. These events must be handled within the context of a single function; Attempting to handle the “toast” message separately is far too likely to result in unreliable synchronization. The REI framework provides two functions that encapsulate the handling of transitions with “toast”:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1">/// Open the indicated view by tapping the specified element, waiting for an expected "toast" message and performing the action it presents.</span>
<span class="c1">/// > Note: If [required] is `true` and the standard load interval expires, the current test will fail with a message indicating this model and the description of the unsatified view load criterion.</span>
<span class="c1">/// - Parameters:</span>
<span class="c1">/// - viewType: Class of the automation model associated with the view that opens after tapping the specified element</span>
<span class="c1">/// - originType: [optional] Class of landing view model if back/close of target view performs unlinked navigation (default = `nil`)</span>
<span class="c1">/// - element: Element that triggers the expected view transition</span>
<span class="c1">/// - fixed: [optional] `true` if the specified element is non-scrollable (default = `false`)</span>
<span class="c1">/// - required: [optional] `false` if incomplete loading of the associated view is allowed (default = `true`)</span>
<span class="c1">/// - Returns: **TransitionWithToast()** object containing target view model and "toast" information; object with default values if view transition fails or "toast" isn't found</span>
<span class="c1">/// - See: ``waitForView``</span>
<span class="c1">/// - See: ``TransitionWithToast``</span>
<span class="c1">/// - See: ``ToastInfo``</span>
<span class="kd">func</span> <span class="n">openViewWithToastAction</span><span class="o"><</span><span class="kt">U</span><span class="p">:</span> <span class="kt">BaseView</span><span class="o">></span><span class="p">(</span><span class="n">_</span> <span class="nv">viewType</span><span class="p">:</span> <span class="kt">U</span><span class="o">.</span><span class="k">Type</span><span class="p">,</span> <span class="nv">originType</span><span class="p">:</span> <span class="kt">BaseView</span><span class="o">.</span><span class="k">Type</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span><span class="p">,</span> <span class="n">byTapping</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">,</span>
<span class="nv">fixed</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="kc">false</span><span class="p">,</span> <span class="nv">required</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="kc">true</span><span class="p">)</span> <span class="o">-></span> <span class="kt">TransitionWithToast</span><span class="o"><</span><span class="kt">U</span><span class="o">></span>
<span class="c1">/// Tap **Back** / **Close** to navigate back to the origin of the current view, as registered in this model, waiting for an expected "toast" message and performing its action if specified.</span>
<span class="c1">/// > Note: This function is unsupported for "navigation hub" view models, as these represent the bottom of the nav stack and therefore have no related **Back** action.</span>
<span class="c1">/// - Parameters:</span>
<span class="c1">/// - element: Element that triggers the expected view transition</span>
<span class="c1">/// - fixed: [optional] `true` if the specified element is non-scrollable (default = `false`)</span>
<span class="c1">/// - required: [optional] `false` if incomplete loading of the associated view is allowed (default = `true`)</span>
<span class="c1">/// - performAction: [optional] `true` if the action presented by the "toast" message should be performed (default = `false`)</span>
<span class="c1">/// - Returns: **TransitionWithToast()** object containing target view model and "toast" information; object with default values if view transition fails or "toast" isn't found</span>
<span class="c1">/// - See: ``origin``</span>
<span class="c1">/// - See: ``waitForView``</span>
<span class="c1">/// - See: ``TransitionWithToast``</span>
<span class="c1">/// - See: ``ToastInfo``</span>
<span class="kd">func</span> <span class="nf">returnToOriginWithToast</span><span class="p">(</span><span class="n">byTapping</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">,</span> <span class="nv">fixed</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="kc">false</span><span class="p">,</span>
<span class="nv">required</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="kc">true</span><span class="p">,</span> <span class="nv">performAction</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="kc">false</span><span class="p">)</span> <span class="o">-></span> <span class="kt">TransitionWithToast</span><span class="o"><</span><span class="kt">T</span><span class="o">></span>
</code></pre></div></div>
<h5 id="toastinfo">ToastInfo</h5>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">struct</span> <span class="kt">ToastInfo</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">message</span><span class="p">:</span> <span class="kt">String</span>
<span class="k">let</span> <span class="nv">identifier</span><span class="p">:</span> <span class="kt">String</span>
<span class="k">let</span> <span class="nv">hasAction</span><span class="p">:</span> <span class="kt">Bool</span>
<span class="p">}</span>
</code></pre></div></div>
<h5 id="transitionwithtoast">TransitionWithToast</h5>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">struct</span> <span class="kt">TransitionWithToast</span><span class="o"><</span><span class="kt">T</span><span class="o">></span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">view</span><span class="p">:</span> <span class="kt">T</span><span class="p">?</span>
<span class="k">let</span> <span class="nv">toastInfo</span><span class="p">:</span> <span class="kt">ToastInfo</span><span class="p">?</span>
<span class="nf">init</span><span class="p">(</span><span class="nv">view</span><span class="p">:</span> <span class="kt">T</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span><span class="p">,</span> <span class="nv">toastInfo</span><span class="p">:</span> <span class="kt">ToastInfo</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span><span class="p">)</span> <span class="p">{</span>
<span class="k">self</span><span class="o">.</span><span class="n">view</span> <span class="o">=</span> <span class="n">view</span>
<span class="k">self</span><span class="o">.</span><span class="n">toastInfo</span> <span class="o">=</span> <span class="n">toastInfo</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>As indicated, the preceding functions handle view transitions that include the appearance of interstitial “toast” messages. The return values of these functions (<strong><code class="highlighter-rouge">TransitionWithToast</code></strong>) encapsulate the view model and a “toast” information (<strong><code class="highlighter-rouge">ToastInfo</code></strong>) object. These are the potential outcomes:</p>
<ul>
<li>If the expected view activates, a reference to the view model is returned in the <em>[view]</em> property.</li>
<li>If the expected view doesn’t activate, the <em>[view]</em> property will be set to <code class="highlighter-rouge">nil</code>.
<ul>
<li><strong>NOTE</strong>: If the <em>[required]</em> argument of the function call is <code class="highlighter-rouge">true</code> (the default value), a test failure will be registered.</li>
</ul>
</li>
<li>If a “toast” message appears, a <strong><code class="highlighter-rouge">ToastInfo</code></strong> object is returned in the <em>[toastInfo]</em> property containing the message text, identifier, and action.</li>
<li>If no “toast” message appears, the <em>[toastInfo]</em> property will be set to <code class="highlighter-rouge">nil</code>.
<ul>
<li><strong>NOTE</strong>: No test failure is registered.</li>
</ul>
</li>
<li>The test itself must determine whether the returned “toast” information (or lack thereof) meets expectations.</li>
</ul>
<p>The following demonstrates the application of <code class="highlighter-rouge">openViewWithToastAction()</code> to tap an element and perform the navigation provided by the “toast” message:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">func</span> <span class="nf">addLabelAndOpenPrintBasket</span><span class="p">()</span> <span class="o">-></span> <span class="kt">TransitionWithToast</span><span class="o"><</span><span class="kt">PrintBasketView</span><span class="o">></span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">optionsSheet</span> <span class="o">=</span> <span class="kt">Element</span><span class="o">.</span><span class="n">options_sheet</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="kt">APP</span><span class="p">)</span>
<span class="k">let</span> <span class="nv">selector</span> <span class="o">=</span> <span class="kt">Element</span><span class="o">.</span><span class="n">add_label</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">optionsSheet</span><span class="p">)</span>
<span class="k">return</span> <span class="nf">openViewWithToastAction</span><span class="p">(</span><span class="kt">PrintBasketView</span><span class="o">.</span><span class="k">self</span><span class="p">,</span> <span class="nv">originType</span><span class="p">:</span> <span class="kt">ToolsHomeView</span><span class="o">.</span><span class="k">self</span><span class="p">,</span>
<span class="nv">byTapping</span><span class="p">:</span> <span class="n">selector</span><span class="p">,</span> <span class="nv">fixed</span><span class="p">:</span> <span class="kc">true</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Note that the “origin” type is specified here. This isn’t always required, but the origin of the Print Basket is the Tools Home view, not the “quick print” menu. Here’s an example of a test that uses this function:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">func</span> <span class="nf">testPrintOptions_PrintBasket</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">sku</span> <span class="o">=</span> <span class="s">"2028570010"</span>
<span class="k">guard</span> <span class="kc">nil</span> <span class="o">!=</span> <span class="n">ascentHomeView</span><span class="o">.</span><span class="nf">launchTheApplication</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">productDetailsView</span> <span class="o">=</span> <span class="n">ascentHomeView</span><span class="o">.</span><span class="n">searchBar</span><span class="o">.</span><span class="nf">searchForSingleItem</span><span class="p">(</span><span class="n">sku</span><span class="p">)</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">quickPrintMenu</span> <span class="o">=</span> <span class="n">productDetailsView</span><span class="o">.</span><span class="nf">tapPrintingOptionsButton</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="k">let</span> <span class="nv">transition</span> <span class="o">=</span> <span class="n">quickPrintMenu</span><span class="o">.</span><span class="nf">addLabelAndOpenPrintBasket</span><span class="p">()</span>
<span class="k">guard</span> <span class="k">let</span> <span class="nv">printBasketView</span> <span class="o">=</span> <span class="n">transition</span><span class="o">.</span><span class="n">view</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="kt">XCTAssertEqual</span><span class="p">(</span><span class="n">transition</span><span class="o">.</span><span class="n">toastInfo</span><span class="p">?</span><span class="o">.</span><span class="n">message</span><span class="p">,</span> <span class="s">"1 label added Print Basket"</span><span class="p">,</span> <span class="s">"Toast message mismatch"</span><span class="p">)</span>
<span class="kt">XCTAssertEqual</span><span class="p">(</span><span class="n">printBasketView</span><span class="o">.</span><span class="nf">getPrintBasketItemCount</span><span class="p">(),</span> <span class="mi">1</span><span class="p">,</span> <span class="s">"Print basket item count mismatch"</span><span class="p">)</span>
<span class="k">guard</span> <span class="kc">nil</span> <span class="o">!=</span> <span class="n">printBasketView</span><span class="o">.</span><span class="nf">backToToolsHome</span><span class="p">()</span> <span class="k">else</span> <span class="p">{</span> <span class="k">return</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This test opens a product page by searching for a SKU, opens the “quick print” menu, adds a label, and opens the print basket via the “toast” action. Synchronization and verification are performed at every step.</p>
<h2 id="summary">Summary</h2>
<p>Applying Page Object Model (POM) patterns to the implementation of XCUITest automation produces well-structured interfaces that are rational and maintainable. Ubiquitous definite synchronization between your tests and the target application ensure reliable, efficient execution. Utilization of framework-supported linked navigation with automatic page-load verification ensures that you always land where you expect - or fail immediately when you don’t. Following the “fail fast with detailed diagnostics” rule eliminates much of the ambiguity that can otherwise make the process of investigating test failures frustrating and time-consuming.</p>
<blockquote>
<p>Written with <a href="https://stackedit.io/">StackEdit</a>.</p>
</blockquote>Scott Babcockscbabco@rei.comCreating Stable, Maintainable User Interface Test Automation in SwiftXCUITest Automation: Encapsulating Element Locators in Swift Enumerations2023-11-29T00:00:00+00:002023-11-29T00:00:00+00:00https://engineering.rei.com/mobile/xcuitest-element-locator-enumerations<h2 id="creating-stable-maintainable-user-interface-test-automation-in-swift">Creating Stable, Maintainable User Interface Test Automation in Swift</h2>
<p>When creating test automation to validate the performance and behavior of a complex software application, adhering to solid engineering practices will produce stable, maintainable tests. This is especially true when driving the application through its user interface.</p>
<p>In user-interface automation, following a page-model approach provides a structured interface that your tests can use to drive the target application and verify expected behaviors. A foundational feature of any user-interface automation strategy is a mechanism to locate and interact with the elements of the interface - buttons, input fields, descriptive text, et cetera.</p>
<p>Apple’s user-interface automation framework for driving iOS application is <a href="https://developer.apple.com/documentation/xctest/user_interface_tests">XCUITest</a>. This framework provides a rich set of classes, properties, and enumerations from which to specify the characteristics of target elements. To produce cohesive models that encapsulate all the details of your element locators, consider Swift enumerations.</p>
<h3 id="defining-element-locators-in-swift-enumerations">Defining element locators in Swift enumerations</h3>
<p>Two common issues that complicate the usage and maintenance of any programming interface are duplicate definitions of constant values and incomplete declarations that require additional context to be used. Each of these characteristics presents challenges when the definitions of associated elements in the target application change, because you’re forced to track down all of the related definitions and apply the same revisions everywhere.</p>
<p>To avoid these sorts of issues, declare element locators in a comprehensive Swift enumeration in the view or component class that’s responsible for interacting with the corresponding elements. For example:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">enum</span> <span class="kt">Element</span> <span class="p">{</span>
<span class="k">case</span> <span class="n">view_title</span>
<span class="k">case</span> <span class="nf">refine_view</span><span class="p">(</span><span class="kt">String</span><span class="p">)</span>
<span class="k">case</span> <span class="nf">facet_label</span><span class="p">(</span><span class="kt">String</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span><span class="p">)</span>
<span class="k">case</span> <span class="nf">filter_cell</span><span class="p">(</span><span class="kt">String</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span><span class="p">,</span> <span class="kt">String</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span><span class="p">)</span>
<span class="kd">func</span> <span class="nf">locator</span><span class="p">(</span><span class="n">_</span> <span class="nv">context</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="o">-></span> <span class="kt">XCUIElement</span> <span class="p">{</span>
<span class="k">switch</span> <span class="k">self</span> <span class="p">{</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">view_title</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">navigationBars</span><span class="p">[</span><span class="n">rawValue</span><span class="p">]</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">refine_view</span><span class="p">:</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">collectionViews</span><span class="p">[</span><span class="n">rawValue</span><span class="p">]</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">facet_label</span><span class="p">(</span><span class="k">let</span> <span class="nv">name</span><span class="p">):</span>
<span class="k">if</span> <span class="k">let</span> <span class="nv">nameOrKey</span> <span class="o">=</span> <span class="n">name</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">predicate</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"identifier == %@ AND (label == %@ OR value == %@)"</span><span class="p">,</span>
<span class="n">rawValue</span><span class="p">,</span> <span class="n">nameOrKey</span><span class="p">,</span> <span class="n">nameOrKey</span><span class="p">)</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">staticTexts</span><span class="o">.</span><span class="nf">matching</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span><span class="o">.</span><span class="n">element</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">staticTexts</span><span class="p">[</span><span class="n">rawValue</span><span class="p">]</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">filter_cell</span><span class="p">(</span><span class="k">let</span> <span class="nv">facet</span><span class="p">,</span> <span class="k">let</span> <span class="nv">filter</span><span class="p">):</span>
<span class="k">if</span> <span class="k">let</span> <span class="nv">facetKey</span> <span class="o">=</span> <span class="n">facet</span><span class="p">,</span> <span class="k">let</span> <span class="nv">filterName</span> <span class="o">=</span> <span class="n">filter</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">cellPredicate</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"identifier == %@ AND value == %@"</span><span class="p">,</span>
<span class="n">rawValue</span><span class="p">,</span> <span class="n">facetKey</span><span class="p">)</span>
<span class="k">if</span> <span class="n">filterName</span><span class="o">.</span><span class="n">isEmpty</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">cells</span><span class="o">.</span><span class="nf">matching</span><span class="p">(</span><span class="n">cellPredicate</span><span class="p">)</span>
<span class="o">.</span><span class="nf">containing</span><span class="p">(</span><span class="o">.</span><span class="n">staticText</span><span class="p">,</span> <span class="nv">identifier</span><span class="p">:</span> <span class="k">Self</span><span class="o">.</span><span class="kt">Constants</span><span class="o">.</span><span class="kt">FILTER_LABEL</span><span class="o">.</span><span class="n">rawValue</span><span class="p">)</span><span class="o">.</span><span class="n">element</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">labelPredicate</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"identifier == %@ AND label BEGINSWITH %@"</span><span class="p">,</span>
<span class="k">Self</span><span class="o">.</span><span class="kt">Constants</span><span class="o">.</span><span class="kt">FILTER_LABEL</span><span class="o">.</span><span class="n">rawValue</span><span class="p">,</span> <span class="n">filterName</span><span class="p">)</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">cells</span><span class="o">.</span><span class="nf">matching</span><span class="p">(</span><span class="n">cellPredicate</span><span class="p">)</span><span class="o">.</span><span class="nf">containing</span><span class="p">(</span><span class="n">labelPredicate</span><span class="p">)</span><span class="o">.</span><span class="n">element</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">return</span> <span class="n">context</span><span class="o">.</span><span class="n">cells</span><span class="p">[</span><span class="n">rawValue</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">var</span> <span class="nv">rawValue</span><span class="p">:</span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">switch</span> <span class="k">self</span> <span class="p">{</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">view_title</span><span class="p">:</span>
<span class="k">return</span> <span class="s">"Search Filter"</span>
<span class="k">case</span> <span class="o">.</span><span class="nf">refine_view</span><span class="p">(</span><span class="k">let</span> <span class="nv">name</span><span class="p">):</span>
<span class="k">return</span> <span class="n">name</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">facet_label</span><span class="p">:</span>
<span class="k">return</span> <span class="s">"RefineSearch.filterFacet.label"</span>
<span class="k">case</span> <span class="o">.</span><span class="nv">filter_cell</span><span class="p">:</span>
<span class="k">return</span> <span class="s">"RefineSearch.filterName.cell"</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">enum</span> <span class="kt">Constants</span><span class="p">:</span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">case</span> <span class="kt">FILTER_LABEL</span> <span class="o">=</span> <span class="s">"RefineSearch.filterName.label"</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The preceding declaration defines a <code class="highlighter-rouge">locator</code> function, a <code class="highlighter-rouge">rawValue</code> computed property, a nested <strong>Constants</strong> enumeration, and four element locator values:</p>
<ul>
<li>A simple locator for an element with a constant identifier…
<ul>
<li>… produces a constant context-relative element locator.</li>
</ul>
</li>
<li>A parameterized locator with a single required associated value…
<ul>
<li>… produces context-relative element locators based on the specified value.</li>
</ul>
</li>
<li>A parameterized locator with a single optional associated value…
<ul>
<li>… [value omitted] produces a constant context-relative locator that finds multiple matching elements.
… OR …</li>
<li>… [value defined] produces context-relative locators based on the defined value that find unique elements.</li>
</ul>
</li>
<li>A parameterized locator with a pair of optional associated values…
<ul>
<li>… [values omitted] produces a constant context-relative locator that finds multiple matching elements.
… OR …</li>
<li>… [first defined, second empty] produces context-relative locators based on the defined value that finds multiple matching elements.
… OR …</li>
<li>… [both defined] produces context-relative locators based on the defined values that find unique elements.</li>
</ul>
</li>
</ul>
<p>Note that the <code class="highlighter-rouge">locator</code> function requires a “context” argument. This is the search context for the returned element locator. The element locator values produced by <code class="highlighter-rouge">locator</code> acquire the identifiers they need from the <code class="highlighter-rouge">rawValue</code> computed property. Associated values are declared or omitted in each context (<code class="highlighter-rouge">locator</code> or <code class="highlighter-rouge">rawValue</code>) as needed.</p>
<p>The “context” argument will typically be the “application” object, which will search the entire view hierarchy. To find elements within a sub-context, provide this instead:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">setFilter</span><span class="p">(</span><span class="n">_</span> <span class="nv">filter</span><span class="p">:</span> <span class="kt">String</span><span class="p">,</span> <span class="k">in</span> <span class="nv">facet</span><span class="p">:</span> <span class="kt">String</span><span class="p">,</span> <span class="n">of</span> <span class="nv">refinement</span><span class="p">:</span> <span class="kt">String</span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">refineView</span> <span class="o">=</span> <span class="kt">Element</span><span class="o">.</span><span class="nf">refine_view</span><span class="p">(</span><span class="n">refinement</span><span class="p">)</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="kt">XCUIApplication</span><span class="p">())</span>
<span class="k">let</span> <span class="nv">filterElem</span> <span class="o">=</span> <span class="kt">Element</span><span class="o">.</span><span class="nf">filter_cell</span><span class="p">(</span><span class="n">facet</span><span class="p">,</span> <span class="n">filter</span><span class="p">)</span><span class="o">.</span><span class="nf">locator</span><span class="p">(</span><span class="n">refineView</span><span class="p">)</span>
<span class="k">if</span> <span class="o">!</span><span class="n">filterElem</span><span class="o">.</span><span class="n">isSelected</span> <span class="p">{</span> <span class="n">filterElem</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>In the preceding example, the “application” object is the context of the “refine” view locator, which defines the context for the “filter” element locator.</p>
<blockquote>
<p><strong>NOTE</strong>: For simple cases, localized handling of sub-contexts like the “refine” view does the trick. When modeling more complex sub-contexts with multiple associated elements and related behaviors, the use of component classes can provide cleaner, more maintainable designs. The details of this structure are documented <a href="xcuitest-page-components.html">here</a>.</p>
</blockquote>
<p>The nested <strong>Constants</strong> enumeration is used to define constant value that are used multiple time within the <strong>Element</strong> enumeration, but are not associated with a defined element locator value. This “constants enumeration” pattern ensures that each value is defined only once, avoiding the risk of retaining stale copies when value updates are needed.</p>
<h2 id="summary">Summary</h2>
<p>The encapsulation of element locators in comprehensive Swift enumerations can eliminate the confusion and inconsistencies that bedevil other approaches. By defining all aspects of each target element in a single place, the task of debugging and maintaining element locators is greatly simplified. With context scoping and associated values, each locator enumeration constants can provide selectors with graduated levels of specificity - from global all-of-type matching to context-constrained unique-instance matching.</p>
<hr />
<blockquote>
<h3 id="note-regarding-another-enum-based-locator-definition-strategy">Note regarding another enum-based locator definition strategy</h3>
<p>While preparing to write this article, I performed a search to determine if others have described similar strategies. I found <a href="https://medium.com/quality-engineering-university/xcuitests-best-practices-for-organizing-locators-with-swift-enumerations-452da45fe7d5">this article</a> by <a href="https://medium.com/@nshthshah">Nishith Shah</a>, which begins with the same basic approach I used. As this strategy is developed through the article, a few key differences emerge:</p>
<ul>
<li>While the use of raw values solves the issue of duplicated constant declarations, this choice precludes the use of associated values which greatly enhance the expressiveness of locators for element collections. While these can be modeled via nested components, this level of factoring often results in a proliferation of tiny classes with limited functionality which are difficult to debug and maintain.<br />
The <code class="highlighter-rouge">rawValue</code> computed property demonstrated in the example above also avoids duplicated constant declarations, with the added benefit that the resulting identifiers can incorporate specified associated values.</li>
<li>Grouping elements by type and handling them in a generic fashion increases complexity and makes the code harder to maintain. The apparent repetition incurred by handling each case independently eliminates the risk of breaking cases you weren’t intending to change when updating cases that require change.</li>
<li>The use of a computed property instead of a function to generate locators precludes the option of providing a search context. While this context is often the application under test, it may also be the root element of a component model (e.g. - an item in a table view). It might also be an entirely different application, such as Safari or Springboard (which is used to access elements in the status bar).</li>
</ul>
</blockquote>
<hr />
<blockquote>
<p>Written with <a href="https://stackedit.io/">StackEdit</a>.</p>
</blockquote>Scott Babcockscbabco@rei.comCreating Stable, Maintainable User Interface Test Automation in SwiftSwitching Trails to SwiftUI: Our Journey at REI2023-10-17T00:00:00+00:002023-10-17T00:00:00+00:00https://engineering.rei.com/mobile/swift-ui<h2 id="declarative-vs-imperative-programming">Declarative vs. Imperative Programming</h2>
<p>When providing instructions for packing a backpack in an imperative framework, you would list each of the <a href="https://www.rei.com/learn/expert-advice/ten-essentials.html">ten essentials</a> and provide exact instructions on locating each item and fitting them all nicely within your pack. In a declarative framework however, you don’t need to meticulously outline these steps. You can instead define the result — a backpack packed with these ten items. The system then figures out how to obtain and fit them in the pack.</p>
<p>This of course, requires you to trust the system to choose the right items and arrange them correctly.</p>
<p>When it comes to computer programming, often there’s little doubt the system will do what you expect. Higher order functions like ‘map’, ‘reduce’ and ‘filter’ give us a taste of declarative programming. Rather than explicitly looping through an array and manipulating data, you can use these methods to declare what you want done and the system will perform the desired manipulation to each element.</p>
<p><img src="/static/images/articles/swift-ui/declaritive-vs-imparitive.png" alt="An example of list manipulation in a declaritive way versus imperative." /></p>
<p><em>The declaritive version of these simple list manipulations is half the length as the imperative version, and easier to follow.</em></p>
<p>When it comes to user interfaces however, there is much more ambiguity in the desired outcome. With that ambiguity comes a reluctance to relinquish control. Can you really trust the system to size things correctly on different form factors without explicit instructions? Will a long list scroll performantly without giving detailed instructions on how and when to create new views versus simply reusing old ones?</p>
<p>Declarative UI frameworks say <em>yes</em> and have been rising in popularity. In 2019 Apple put out their own declarative UI framework — SwiftUI.</p>
<h2 id="deciding-to-declare">Deciding to Declare</h2>
<p>Like most iOS developers, we approached SwiftUI with some trepidation here at REI. The framework is still evolving and unlike longer lived frameworks, the answers to questions that arise might not yet be chronicled on StackOverflow. The benefits however are clear. The concise, descriptive SwiftUI code makes maintaining, refactoring, and reusing components much simpler than it was before. It also seems clear that it’s Apple’s preferred framework, with components such as Home Screen Widgets requiring SwiftUI. Navigating the SwiftUI learning curve seems inevitable for iOS development, it’s only a question of when.</p>
<p>With this in mind, we recently decided to move to SwiftUI for our retail employee app, Ascent. This was an easier decision for this app than most, as we have the luxury of owning the devices the app is deployed on. We can ensure everyone is on the latest iOS version, allowing us to use latest and greatest features of SwiftUI as they become available. The roadmap for the app also lended itself nicely to an overhaul. As we continued to add features to the app, it required a redesign of the fundamental navigation structure. This was the perfect opportunity to build a foundation of SwiftUI.</p>
<h2 id="navigating-the-move-to-swiftui">Navigating the Move to SwiftUI</h2>
<p>We planned for all new development to be in SwiftUI, and to refactor existing screens to SwiftUI when meaningful changes need to be made. To set ourselves up for success, we adopted a SwiftUI architecture that would make refactoring existing UIKit screens as painless as possible.</p>
<p>Our UIKit architecture follows the <a href="https://saad-eloulladi.medium.com/ios-swift-mvp-architecture-pattern-a2b0c2d310a3">Model, View, Presenter.</a> (MVP) pattern. These presenters contain the business logic that setup the data (model) needed to a particular screen (view), and then communicates the necessary updates to the view. The communication between presenter and view are backed by protocols, allowing mock views for easy unit testing.</p>
<p>When refactoring an existing UIKit using MVP to SwiftUI, we were able to keep the presenters largely unchanged. The view however is replaced by a new SwiftUI view object, composed of one or many subviews. Each subview defines a ViewModel for anything that can change in it. Then, we can add these ViewModel objects as @Published properties to our presenters, and reference them as @State properties in the top level SwiftUI view. Instead of presenter invoking the protocol methods on the view, we simply update the viewModels to the desired state when appropriate. This ends up looking a lot like a <a href="https://www.hackingwithswift.com/books/ios-swiftui/introducing-mvvm-into-your-swiftui-project">Model, View, ViewModel</a> architecture.</p>
<p><a href="/static/images/articles/swift-ui/AscentSection.png"><img src="/static/images/articles/swift-ui/AscentSectionSmall.png" alt="Example of ViewModel and it's use in a Presenter" /> </a></p>
<p><em>Our AscentSection component, its use in the ToolsPresenter as @Published properties, and resultant screen.</em></p>
<p>The result is very clean and modular view code that naturally lends itself to reusable components. We also retain testability for the business logic in our presenters. Instead of setting up mock views, we simply observe the @Published properties in each presenter and listen for the expected updates.</p>
<p><img src="/static/images/articles/swift-ui/unit-test.png" alt="An example of unit testing a @Published property" /></p>
<p>Developing in this environment can be a joy. Instead of being bogged down by boilerplate code and view lifecycles, one can focus on what really matters-— the look and feel of the app and the associated business logic. But unfortunately this isn’t always the case…</p>
<h2 id="bumps-on-the-trail">Bumps on the Trail</h2>
<p>While the end result is clean, modular views, getting there can sometimes be a struggle. Simple things, like setting padding around a view can have one scratching your head until you realize that the order of applying View Modifiers matters. When you add them to a view, you’re not simply setting properties on an object as in UIKit, but as the name implies, <a href="https://www.swiftbysundell.com/questions/swiftui-modifier-order/">you’re actually modifying the views</a>. Luckily, despite SwiftUI being a newer framework, most issues like this are in fact Google-able, such as why a <a href="https://stackoverflow.com/questions/57191013/swiftui-cant-tap-in-spacer-of-hstack">row with a Spacer element isn’t registering tap events as you’d expect</a>.</p>
<p>The real struggle occurs when SwiftUI just doesn’t want to do that which you declare. We faced one such frustrating issue trying to allow users to switch their keyboard type while typing. In UIKit, this was as simple as adding toolbar buttons to the keyboard and setting the keyboard type on the UITextField. SwiftUI however handles a similar setup differently. Instead of changing the keyboard type while the keyboard remains on the screen, SwiftUI sees fit to dismiss the keyboard once changes to the type are made, requiring another invocation of editing mode for the new type to appear. This wouldn’t be too big of a deal as we can programmatically invoke the keyboard again with @FocusState — except this does not work in our case where a <a href="https://stackoverflow.com/questions/74245149/focusstate-textfield-not-working-within-toolbar-toolbaritem">textfield is within a toolbar</a> and you’re not at the top level of a NavigationStack.</p>
<p>In situations like this it can be very hard to determine if the problem is you, or a limitation of SwiftUI. Others were reporting similar issues here, and we ended up taking the advice posted on Stack Overflow and the Apple Developer Forums… Use the imperative UITextField wrapped in a UIViewRepresentable to achieve what we had before. With our philosophy of “everything new in SwiftUI”, this seemed like a step backwards. We want to set our app up for the future. Adding new UIKit components seemed like taking on technical debt, but in the current state of SwiftUI you do need to make some compromises – either on functionality or technical decisions.</p>
<p>Another challenge is performance. In the world of imperative programming, you can see where views are created and laid out, catching bottlenecks that may slow your app. Such details are abstracted away in declarative frameworks. This can lead to pitfalls where an update to a single textField can cause a cascading redraw of all views if you’re using a single view model for an entire screen as opposed to distinct components each with their own view model. In other cases, SwiftUI views don’t always do quite what you’d think they should. One could reasonably expect a SwiftUI List to act as a performant UITableView would, caching views and updating their content instead of constantly creating new views when scrolling. This however does not seem to be the case. When we used a List to display a large amount of search suggestions, we saw laggy non-performant scrolling. For <a href="https://developer.apple.com/documentation/swiftui/creating-performant-scrollable-stacks">performant scrollable lists</a>, Apple recommends using a LazyVStack inside of a Scrollview instead. This did work well for us, but it does not seem very intuitive and was not the first thought for fixing our issues.</p>
<h2 id="looking-back">Looking Back</h2>
<p>Our journey transitioning to SwiftUI was both rewarding and challenging. Over 70% of our Ascent’s views are now in SwiftUI. With a couple more releases we’ll get close to 100%. As components are built and knowledge in the framework is gained, the path ahead gets smoother—- both in developing new features and in maintaining and changing existing ones.</p>
<p>Despite the challenges along the way, the journey has been a success. This is in no small part due to the support of the entire team. Our product owner saw the value in adopting the new technology and supported the journey even though it meant less accurate estimates and an occasional change to better accommodate what’s possible in SwiftUI. Our designer composed new features with a set of building blocks, lending themselves extremely well to modular, reusable SwiftUI views. Our testers put the app through the wringer, finding easily overlooked issues and updating UI tests to work with the new layouts. Our project manager kept us focused and moving in the right direction while our backend engineer ensured we didn’t have to struggle to get the data we need. And our architect gave us the nudge to go blaze this trail, giving backup when things got challenging.</p>
<p>Looking back, we are now left with the concise, maintainable view code we were looking for. We’ve also gained vaulable experience in SwiftUI and declarative frameworks that will servce us well in the future. With Android and other platforms also adopting declarative frameworks, the specifics of each platform are becoming less and less important. Perhaps some day in the near future we won’t have to worry about platform dependant view code at all– we’ll be able to simply declare how we want the app to look across platforms.</p>
<p><img src="/static/images/articles/swift-ui/packing.jpg" alt="" /></p>
<p>So, can you trust SwiftUI to pack your backpack? Yes, but make sure to double check the result and be willing to provide some imperative steps if the batteries on your flashlight need changing.</p>Ed Williamsedwilli@rei.comDeclarative vs. Imperative ProgrammingAutomated Accessibility Testing: From Passive to Preventative2023-10-11T00:00:00+00:002023-10-11T00:00:00+00:00https://engineering.rei.com/frontend/automated-accessibility-testing<p><em>Implementing and scaling automated accessibility testing on REI.com</em></p>
<p>At REI, we believe that time outside is a fundamental right that all should be able to enjoy. This commitment to inclusion doesn’t start and end with marketing. It’s something we try and live up to every day as we design and engineer new customer features and experiences on REI.com.</p>
<p>Back in 2021, like many other companies, we were stuck in an endless cycle of <em>audit & remediate</em> with our digital accessibility efforts — reacting after-the-fact to the barriers we were putting in front of our customers. Recognizing a need to act sooner, we set out to engineer an automated solution that would be:</p>
<ul>
<li>Effective — limited in its false positives.</li>
<li>Scalable — easily translatable to any microsite environment within our common framework.</li>
<li>Passive — require minimal human intervention to sustain.</li>
<li>Blocking — not only advising on accessibility issues but preventing them from impacting people in the first place.</li>
</ul>
<p>Now two years later, we’ve successfully built and scaled a suite of automated tests that are consumed by over 90% of our customer-facing applications, testing thousands of deployments per year for common accessibility missteps, and blocking inaccessible code from seeing the light of day.</p>
<p>We know automation is only 30% of the accessibility testing puzzle, because not everything can be automated today, but we’re proud of the foundation we’ve built and the lessons we’ve learned along the way.</p>
<figure class="figcaption">
<img class="figcaption__image" src="/static/images/articles/automated-accessibility-testing/figure-1.png" alt="screen capture of Jenkins dashboard graph, described in caption" />
<figcaption class="figcaption__caption">
<p><em>Figure 1: Accessibility Test Count - April '22 to Today</em></p>
<p>The graph above shows the benefits of integrating accessibility tests into a common framework, instead of relying on a team-by-team approach — rapid adoption. In October 2022, we were running less than 20 tests per day. Today, we're consistently over 300 with a greater than 80% pass rate.</p>
</figcaption>
</figure>
<h2 id="building-for-scale">Building for Scale</h2>
<p>From the beginning, our most important goal was enabling scalability. In talking to individual employees, we learned that accessibility testing often feels intrusive and additive to the day-to-day work of product teams. We wanted a solution that could be a seamless part of the product development process. Our test suite works by minimizing friction and maximizing positive outcomes for both employees and our customers.</p>
<p>Another reason that scalability was important is due to the rate of change happening on REI.com. With the <a href="https://engineering.rei.com/devops/how-we-built-a-microservices-platform.html">decomposition of our code monolith</a> into individual microsites and services underway, and due to the nature of our continuous delivery model, new applications are popping up all the time. It’s hard for our small accessibility team to keep up with the maintenance work.</p>
<p>Luckily, many of the customer-facing web applications are built on our in-house framework, Crampon. A common framework allows us to take advantage of pre-existing templating, baking in accessibility best practices and testing infrastructure from day one. It also means that as we started our automated testing journey in 2021, our existing web applications were backwards compatible with our new ideas.</p>
<p>To put in perspective just how easy it is now for developers to implement basic accessibility testing in their application today, I’ve included the steps below for a new or existing Crampon application:</p>
<p>For new applications:</p>
<ol>
<li>A new application is created via Chairlift, our internal application provisioning service.</li>
<li>The developer gives the application a name and selects <em>Crampon Application</em> to create a Crampon-based microsite or microservice.</li>
<li>The developer chooses <em>Includes microsite UI</em></li>
<li>After completing the remaining steps in the set-up wizard, the developer now has a repository with our starter application that includes the latest version of Crampon, which includes an accessibility testing starter template by default.</li>
<li>All that’s left to do is specify the URLs to test and implement the <code class="highlighter-rouge">isCoOpA11yCompliant</code> method into their existing Selenium-based system tests. These can be written to support tests for basic static content or more advanced applications can include dynamic page states and conditions.</li>
</ol>
<p>For existing applications:</p>
<ol>
<li>A new change to the test suite has been introduced in the Crampon repository.</li>
<li>A new version of Crampon has been released.</li>
<li>Existing microsites upgrade their applications to the newest version of Crampon.</li>
<li>All that is left to do is specify the URLs to test and implement the <code class="highlighter-rouge">isCoOpA11yCompliant</code> method into their existing Selenium-based system tests. These can be written to support tests for basic static content or more advanced applications can include dynamic page states and conditions.</li>
</ol>
<p>As you can see, we have taken much of the work of understanding accessibility and test libraries out of the equation and let test engineers do what they do best, write tests.</p>
<h2 id="ensuring-effectiveness">Ensuring effectiveness</h2>
<p>With scale comes challenges related to the effectiveness of our test suite. A one-size-fits-all solution isn’t tremendously effective, especially with 22 unique applications serving customer functions from landing page to checkout. Not to mention how diverse our product teams are in their approaches to writing acceptance criteria, code, and tests.</p>
<p>Our first step was to decide on a base test library that could limit false positives while remaining flexible enough to conform to each unique application in a way that was seamless and performant. After extensive testing, we landed on <a href="https://github.com/dequelabs/axe-core">axe-core</a>. It’s free, open source, something we were familiar with already, and allowed us to add/remove rules and standards easily.</p>
<p>Then, after careful consideration, we narrowed down our current ruleset to just 42 rules. We intentionally exclude page elements that are included as part of our base template to eliminate test noise and ensure our suite is performant.</p>
<p>With a solid ruleset in place, our next move was to exclude the header and footer navigation for testing and only test content within the core page content for each application. Our navigation is persistent across many pages, and we found that choosing to test this code once in isolation, compared to testing it within every application removed a lot of noise caused by test failures, allowing teams to work quicker.</p>
<p>Accessibility tests run against our navigation (header and footer), as well. Our Header/Footer exists as its own microsite, which makes testing it easier.</p>
<p>Finally, we needed to make it easy to make our tests blocking for deployments. We want to prevent issues from impacting customers. To do that in a way that’s durable, we needed to ensure tests are always passing, by accounting for the code we don’t own — 3rd party vendor content. Without doing this, 3rd parties could release inaccessible code and prevent our developers from pushing to production.</p>
<p>We solved this problem by developing and implementing a change to our test suite via Crampon that enabled developers to skip testing for certain selectors within their UI.</p>
<p>We rolled this out globally by working with individual teams to upgrade to the most recent version of our common framework. And with our implementation in place, teams are easily and efficiently able to understand how well their application does at passing automated accessibility tests, without the noise that navigation and vendor solutions might add.</p>
<h2 id="ushering-in-preventative-measures">Ushering in preventative measures</h2>
<p>With accessibility tests running on every microsite and in many cases passing, we could have stopped there. However, we were determined to do more than tell people their code had accessibility issues.</p>
<p>We wanted to prevent the negative impacts of inaccessible code before it became reality.</p>
<p>Initially our test suite was running in production only. In theory, we could have flipped the Boolean value from false to true and prevented people from shipping code that didn’t pass the test. This would have been a bit heavy-handed and potentially added additional friction when folks were on a tight deadline or in a code-freeze situation.</p>
<p>Instead, we chose to implement an identical set of tests in each team’s staging environment. This acts as a proxy for blocking tests with less risk to deployments since teams test their work in staging before pushing to production. So far, this has been effective in preventing regression without negatively impacting our teams’ workflows.</p>
<h2 id="road-to-maturity">Road to maturity</h2>
<p>Now you’re wondering, <em>how did you scale so quickly?</em> and much of that success can be attributed to our maturity plan, illustrated in the graphic below by our Principal Accessibility Engineer, <a href="https://engineering.rei.com/authors.html#Harmony%20Hames">Harmony Hames</a>.</p>
<p><strong><em>Note</em></strong>: For the purpose of this article you can think of staging environment and try pipelines interchangeably. This is pre-production code.</p>
<figure class="figcaption">
<img class="figcaption__image" src="/static/images/articles/automated-accessibility-testing/figure-2.png" alt="screen capture of flow chart, described in text below" />
<figcaption class="figcaption__caption">
<p><em>Figure 2 - Illustrating how REI.com applications implemented our test suite</em></p>
</figcaption>
</figure>
<p>Getting teams to prioritize accessibility testing wasn’t easy, and it wasn’t an overnight success. The choices we made and the features (like <em>exclude</em> functionality) we developed were the result of a lot of trial and error. It took us almost 2 years to get to the stage we’re at now. That said, we chose to ease teams into change vs. forcing a lot at once as illustrated above:</p>
<ol>
<li>Upgrade to the latest version of our framework, Crampon</li>
<li>Ensure you’re using the latest test method.</li>
<li>Get your production tests configured correctly if they’re not.</li>
<li>Get your production tests passing.</li>
<li>Implement tests in your staging environment to prevent regression.</li>
</ol>
<p>At any given time, you’d have 2 teams in stage one, another few with passing tests, and a handful of others not on the latest version of Crampon. It felt like chaos but having a well-documented plan for a team to mature their practice and only 1-2 things to focus on at a time, provided just the right amount of motivation for teams to achieve what looked impossible.</p>
<h2 id="finding-and-closing-the-gaps">Finding and closing the gaps</h2>
<p>The tests are running on most user experiences, passing, and they’re blocking. We’re done, right? Not quite.</p>
<p>Though the reality of the situation was that we’re running over 400 tests, split between staging and production, and most of REI.com is templated, we found that they were only as good as the templates we were covering. There were page types we weren’t covering with any accessibility tests.</p>
<p>To solve this problem, we needed to understand where the gaps were, and then improve each application’s tests to achieve optimal coverage. To do that, we needed to drastically expand the scope of the tests, since today our problems were outside of our existing URL set.</p>
<p>One, albeit cumbersome, way to determine which page types weren’t being covered, would be to browse them like a user would and run a browser extension (in our case axe DevTools, since it’s an identical test library to our pipeline tests). If there were test failures, it meant it contained failures due to rules we’d skipped, or it wasn’t tested.</p>
<p>Based on a set of 2,600 pages which we believed would cover all page types, we estimated this route would take an estimated 86.6 hours to complete (2 min x 2,600 pages / 60 min). This is way too long, and way too cost prohibitive, especially if we wanted a solution that would keep up with our site’s pace of change.</p>
<h2 id="arborist">Arborist</h2>
<p>Enter Arborist, our homegrown automated solution to doing the work described above. This key piece of technology unlocked several benefits for us as we reached for the best possible test coverage:</p>
<p><strong>Increased visibility (9000% larger scope)</strong></p>
<ul>
<li>3rd party audits of REI.com are especially limited in scope due to cost.</li>
<li>Our pipelines are testing an estimated 400 pages.</li>
<li>Arborist tests around 2,600 pages.</li>
</ul>
<p><strong>Increased efficiency (-94% reduction in time to results)</strong></p>
<ul>
<li>A human triggering the automated tests via a browser extension (axe DevTools, Google Lighthouse, etc.) would have taken an estimated 86.6 hours (2 min x 2,600 / 60 min) to complete a run of 2,600 pages.</li>
<li>Arborist runs for an average of only 4 hours and 32 minutes.</li>
</ul>
<p><strong>Increased frequency</strong></p>
<ul>
<li>A human might test pages only as time and bandwidth allows.</li>
<li>Arborist runs on REI.com every Friday.</li>
</ul>
<p><strong>Increased flexibility</strong></p>
<ul>
<li>Test libraries, people, and standards change. With an extra tool that runs outside of our deployment pipelines, we can now test new enhancements and rule sets before impacting product teams with global changes.</li>
</ul>
<p>Since we’ve launched Arborist, we’ve been able to dial in our test coverage and improve upon the work we’d already worked with teams and applications to adopt. Additionally, we’re able to get more data, more often, something that’s been a challenge in the accessibility space.</p>
<figure class="figcaption">
<img class="figcaption__image" src="/static/images/articles/automated-accessibility-testing/figure-3.png" alt="screen capture of flow chart, described in text below" />
<figcaption class="figcaption__caption">
<p><em>Figure 3 Each run of Arborist has shown a notable decline in automated accessibility issues, indicating we're not regressing, and highlighting what's remaining due to 3rd party vendors.</em></p>
</figcaption>
</figure>
<h2 id="conclusion">Conclusion</h2>
<p>Though we still have a lot of work to do, knowing that automated accessibility testing can only really prevent 30-50% of issues, and that manual testing is required, too, we’re proud of the testing infrastructure we’ve built. We hope to close the remaining gaps with the manual testing standards we’re piloting now.</p>
<p>What was once a culture of annual audits and remediation, is now one that regularly checks in on whether their accessibility tests are passing. We’re actively preventing accessibility issues from reaching customers, leading to a more accessible and friendly shopping experience.</p>
<p>Putting people over profit, so all people can enjoy time outside.</p>Cooper Hollmaierchollma@rei.comImplementing and scaling automated accessibility testing on REI.comIntroduction to Chaos Engineering2023-06-14T00:00:00+00:002023-06-14T00:00:00+00:00https://engineering.rei.com/devops/chaos-engineering<p><em>Does the thought of being oncall for a critical service keep you up at night? (Or do bears?)</em></p>
<h2 id="customer-scenario">Customer scenario</h2>
<p>Let’s say you had a great day walking around the zoo. Say hello to <a href="https://www.nwtrek.org/animals/bears/grizzly-cubs/">Huckleberry</a>. Or is it Hawthorne? Does it matter? Look at those claws!</p>
<p><img src="/static/images/articles/chaos-engineering/grizzly.jpg" alt="An image of a standing grizzly bear" /></p>
<p>The thought of meeting either one in the wild keeps you up all night, so you decide to do some shopping.</p>
<p><img src="/static/images/articles/chaos-engineering/shoppingcart1.png" alt="REI.com shopping cart with bear spray and bear can" /></p>
<p>Clicking submit order and ready to go back to bed. Wait a minute - didn’t go as planned?</p>
<p><img src="/static/images/articles/chaos-engineering/shoppingcart2.png" alt="Shopping failed to place order" /></p>
<h2 id="engineer-scenario">Engineer scenario</h2>
<p>You’re the oncall engineer supporting REI’s shopping cart & checkout and you get an alert in the middle of the night that some service (xxxService) is throwing errors. After dialing into the major incident conference call you are asked what is the customer experience when xxxService is unavailable? Panic time, right? Good thing the camera is off!</p>
<p>REI’s Checkout microservice orchestrates numerous dependencies to provide shopping cart and checkout functions for rei.com and mobile apps. For example, Customer Info, Dividend, Gift Card, Credit Card, Coupons, etc. With so many dependent services, it’s inevitable that something will break at some point. The goal is to minimize the customer impact during checkout.</p>
<p><em>Purposefully fuzzy relationship image</em>
<img src="/static/images/articles/chaos-engineering/relations.png" alt="Purposefully fuzzy relationship image" /></p>
<h2 id="chaos-engineering-to-the-rescue">Chaos Engineering to the rescue?</h2>
<p>Chaos testing is just one of the many types of testing we do at REI to ensure our services are working properly.</p>
<p>From <a href="https://principlesofchaos.org/">Principles of Chaos Engineering</a>:</p>
<blockquote>
<p>Chaos Engineering is the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.</p>
</blockquote>
<p>There are many types of tests that you can do that could fall under the umbrella of Chaos Engineering. For the purposes of this article, I have focused on testing individual services going down.</p>
<h3 id="game-day-scenario">Game day scenario</h3>
<p>This works best with the whole team present: Developers (frontend & backend), architects, SDETs, product managers, etc.</p>
<ul>
<li>Hypothesis - What happens when the xxxService is unavailable?</li>
<li>Test - Disable xxxService. Since all of our services are invoked using the <a href="https://resilience4j.readme.io/docs">Resilience4j</a> circuit breaker, this is achieved by putting the breaker into “Open” state. We did this in our test environment. You could also change the read timeout to 1ms, or use Wiremock or something similar to respond with a 5xx error. Side note - we’ve migrated from Hystrix to Resilience4j. See excellent article from Jeremiah Pierucci <a href="https://engineering.rei.com/site-reliability/hystrix-circuit-breakers.html">here</a>.</li>
<li>Observe - Did it behave the way you expected? While going through the exercise, sometimes the hypothesis did not match what we observed, and we needed to make some change to address it. In other cases, the website did not behave the way we wanted it to. Doing this live with the whole team was very enlightening for me, as we sometimes had different ideas of how the site should behave (For example, what happens when xxxService is not available - should it block customers from submitting their order? What should the user experience be?)</li>
<li>Fallback - do we want to add a fallback response when any of the services are down?</li>
<li>Document - With so many dependencies it is important to document the expected behavior (otherwise I won’t remember at 2 am when I am oncall)</li>
</ul>
<h2 id="continuing-the-chaos-journey">Continuing the Chaos journey</h2>
<p>This testing is not intended to be done one time only - you will need to be diligent when new services are added, and it would make sense to repeat these tests on a semi-regular basis, as other parts of the code could have changed.</p>
<p>The experiment described above can be thought of as a building block of Chaos Engineering. As you gain confidence in this testing, you could increase scope to include things like database, cloud infrastructure, spikes in traffic, or really anything you can think of that could disrupt your service. Eventually, as your Chaos Engineering practice matures, you could automate these tests and add them to your build pipeline, and perhaps eventually test in production.</p>
<p>Hopefully, the result is that both customers and oncall engineers sleep well at night. You do have robust monitoring and alerting for your application, right?</p>
<p><img src="/static/images/articles/chaos-engineering/success.png" alt="Order successfully placed" /></p>Tom Girardtgirard@rei.comDoes the thought of being oncall for a critical service keep you up at night? (Or do bears?)Accessibility + content design: a powerful combo to make you a better developer2023-03-06T00:00:00+00:002023-03-06T00:00:00+00:00https://engineering.rei.com/frontend/accessibility-plus-content-design<p><em>A silly story that helps illustrate the benefits of understanding both accessibility and content design.</em></p>
<h2 id="introduction">Introduction</h2>
<p>If you didn’t learn about accessibility when you were learning development, you’re not alone. Even today it’s rare to find a development course offering more than a passing mention.</p>
<p>Writing accessible code, at its core, means writing code correctly. When combined with accessible <a href="https://contentdesign.london/blog/what-is-content-design">content design</a>, it will make your site generally more usable.</p>
<p>But more importantly, it enables people with disabilities to experience, and use the internet. In the world we live in today, that is unquestionably a human right.</p>
<p>Learning accessibility is no small task. There is much to learn, but it can be broken up into phases. Many certification courses will provide study outlines, or you can create your own. No matter how you choose to learn, it is well worth the effort.</p>
<p>Here are just a few of the basics:</p>
<p><strong>You’ll learn about <a href="https://www.w3.org/TR/WCAG21/">WCAG (Web Content Accessibility Guidelines)</a></strong>, which will give you:</p>
<ul>
<li>A framework to help you understand the basic concepts of accessible web content.</li>
<li>Success Criteria that provide techniques and a framework for testing your work.</li>
<li>A deeper understanding of how an inaccessible web impacts people with disabilities.</li>
</ul>
<p><strong>You’ll learn about <a href="https://www.w3.org/TR/wai-aria-1.1/">ARIA (Accessible Rich Internet Applications)</a></strong>, which has:</p>
<ul>
<li>A library of existing patterns for web components in the <a href="https://www.w3.org/WAI/ARIA/apg/">ARIA Authoring Practices Guide (APG)</a>.</li>
<li>The expected keyboard patterns for these components.</li>
<li>Best practices around designing keyboard patterns for novel components.</li>
<li>Roles, states, and properties of elements on the web.</li>
</ul>
<p><strong>You will dive deeper into <a href="https://web.dev/learn/html/semantic-html/">semantic HTML</a></strong>, and understand why it is critical for those using assistive technologies.</p>
<p>Some examples of how this knowledge will benefit you as a developer:</p>
<ul>
<li><strong>Your perspective will be more inclusive.</strong> Not all users use a mouse. Not all users can see the screen. You’ll move away from this “default thinking.”</li>
<li><strong>Your code will be more performant.</strong> Semantic and well-structured code requires less scripting and fewer DOM nodes.</li>
<li><strong>Your code will be SEO-friendly.</strong> Using semantic elements and avoiding duplication of content is good for SEO rankings.</li>
<li><strong>You’re less likely to create device-specific bugs.</strong> Alternative input methods will be top-of-mind. Using built-in browser functionality is more reliable than custom coding.</li>
<li><strong>You will be more confident.</strong> You’ll learn to quickly break down designs into known components and patterns.</li>
<li><strong>You will be more efficient.</strong> You’ll see design issues in refinement, instead of bending over backward to fix them in code.</li>
<li><strong>You will be more employable.</strong> This <a href="https://www.mdpi.com/2227-9709/7/1/8">case study</a> notes that “A study reported that web designers with competence in solving accessibility problems are more employable.”</li>
</ul>
<h2 id="how-content-design-thinking-and-accessibility-work-together">How content design thinking and accessibility work together</h2>
<p>Content design is about determining the best way to present the info people need to get stuff done—before getting into detailed interface design. Once the content and story interactions are clear, the design should center around that story flow and reinforce it.</p>
<p>Developers can play a role in this by helping to preserve the meaning of content when they see it getting lost in design translation. It can be a strong habit to look at a visual design and immediately start converting it into a layout. But by seeing a design first as a content outline, it becomes easier to recognize potential issues.</p>
<p>Many of the skills you gain learning accessibility support this kind of thinking. We’ll use an example to demonstrate this in action.</p>
<p>Starting with a visual design, we’ll examine how the average developer might start to break this down conceptually.</p>
<h2 id="a-story-of-space-creatures">A story of space creatures</h2>
<p>For this mental exercise, we’ll put together a fake team:</p>
<ul>
<li><strong>Ted:</strong> The designer. He’s new on the job, but he’s doing his best.</li>
<li><strong>Chadwick:</strong> a traditionally trained front-end developer.</li>
<li><strong>Allie:</strong> a front-end developer who trained in accessibility best practices.</li>
</ul>
<p>We’ll pretend this design contains actual content and not Forcem Ipsum.</p>
<p>This is how the design came in from Ted.</p>
<figure class="figcaption">
<img class="figcaption__image" style="max-width: 250px; border: none; border-radius: 0;" src="/static/images/articles/accessibility-plus-content-design/designer-spec_mobile.png" alt="screen capture of the mobile version of the site, described in caption" />
<figcaption class="figcaption__caption">
<p><strong>Shown above:</strong> The mobile version of the layout,
showing the content in understandable reading order. The full text of the layout is below, but it is not necessary to read it to understand this article (maybe just to get a laugh).</p>
<details class="expander">
<summary class="expander__button">Show full image text</summary>
<div class="expander__text">
<p>Note: the heading levels are given here based on appearance (size).</p>
<p>Image of a child (will be described below, no spoilers!)</p>
<p>Heading level 1: Ewoks and ET: A Risky Mix</p>
<p>The approach will not be easy. You are required to maneuver straight down this trench and skim the surface to this point.</p>
<p>The shaft leads directly to the reactor system. A precise hit will start a chain reaction which should destroy the station. Only a precise hit will set up a chain reaction. The shaft is ray-shielded, so you’ll have to use proton torpedoes.</p>
<p>May the Force be with you!</p>
<p>Heading level 2: Speeder bikes AND flying bicycles?</p>
<p>At least the information in Artoo is still intact. What's so important? What's he carrying? The technical readouts of that battle station. I only hope that when the data is analyzed, a weakness can be found. It's not over yet! It is for me, sister!</p>
<p>She lied! She lied to us! I told you she would never consciously betray the Rebellion. Terminate her...immediately! Stand by, Chewie, here we go. Cut in the sublight engines. What the...? Aw, we've come out of hyperspace into a meteor shower.</p>
<p>Heading level 3: Speeder bike technology</p>
<p>You're sure the homing beacon is secure aboard their ship? I'm taking an awful risk, Vader. This had better work. Not a bad bit of rescuing, huh? You know, sometimes I even amaze myself. That doesn't sound too hard. Besides, they let us go. It's the only explanation for the ease of our escape. Easy...you call that easy? Their tracking us! Not this ship, sister.</p>
<p>Image of a wookie (again, description below, no spoilers).</p>
<p>Heading level 2: What about Wookiees?</p>
<p>They'll have that shield down on time.</p>
<p>Earned it, I have. Master Yoda, you can't die. Strong am I with the Force. but not that strong! Twilight is upon me and soon night must fall.</p>
<p>He will come to you and then you will bring him before me. As you wish. Luke! Luke! Oh, Master Luke. I'm afraid that Artoo's sensors can find no trace of Princess Leia.</p>
<p>Wait, Leia! Quick! Jam their comlink. Center switch! Hey, wait! Ahhh! Move closer! Get alongside that one! Get him! Keep on that one! I'll take these two!</p>
</div>
</details>
</figcaption>
</figure>
<figure class="figcaption">
<img class="figcaption__image" style="border: none; border-radius: 0;" src="/static/images/articles/accessibility-plus-content-design/designer-spec_desktop.png" alt="screen capture of the desktop version of the site, described in caption" />
<figcaption class="figcaption__caption">
<p><strong>Shown above:</strong> The desktop version of the layout.</p>
<p>In this version, the content appears to be in two columns, and
the order has changed. Now "What about Wookiees?" comes right after the opening section, and the image of the Wookie has moved to the end of the content.</p>
</figcaption>
</figure>
<p>When the team meets to review the design before development, Ted asks if there is anything that needs clarification.</p>
<h3 id="chadwicks-review">Chadwick’s review</h3>
<p>Chadwick thinks about the visual design and how it compares between desktop and mobile. He looks at which elements shift, and thinks about how to do this with CSS. He notes a few things:</p>
<ul>
<li>It’s clearly a column design on desktop.</li>
<li>The elements are in a different order between desktop and mobile. He thinks through ways to handle that:
<ul>
<li>CSS Grid? Maybe. He could rearrange the content with grid-template-areas. But with the column design, he would likely have gaps as the page resized. Any answer to that is likely to be quite fussy, but possible.</li>
<li>He could duplicate the content and use CSS to show/hide the areas at different breakpoints.</li>
<li>He could use a buffered resize listener to move the content on page resize.</li>
</ul>
</li>
<li>Other than that, it’s just basic elements - heading, text, and images.</li>
</ul>
<p>Chadwick asks Ted asks a few questions about how the page layout changes at different breakpoints. Other than that, he thinks he’s got it.</p>
<h3 id="allies-review">Allie’s review</h3>
<p>Allie takes a content-first, accessibility-focused approach. For this design, she considers:</p>
<ul>
<li>Semantic HTML and reading order.</li>
<li>The way assistive technology will interact with the page.</li>
<li>Other ways that users might experience the page, or change the display.</li>
</ul>
<p>This brings a few things to light:</p>
<ol>
<li>There is non-text content (images) that will need alternative text for those who can’t see the screen.</li>
<li>The banner under the “What about Wookiees” heading only exists on desktop.</li>
<li>The design dictates a change in reading order.</li>
</ol>
<p>She addresses these issues one at a time.</p>
<h4 id="1-missing-alternative-text">(#1) Missing alternative text</h4>
<p>She asks about #1 (alternative text for the images) first. Ted quickly scratches out some alternative text, and their project manager adds it to the ticket:</p>
<ul>
<li><strong>Image 1:</strong> “An eight-year-old child with a bowl haircut, wearing pink corduroy pants and a baseball shirt with blue arms and E.T. on the front. She stands holding a cardboard cutout of an Ewok, in front of a Star Wars backdrop.”</li>
<li><strong>Image 2:</strong> “A line sketch of a Wookiee, by an artist who is clearly quite skilled.”</li>
</ul>
<h4 id="2-content-exists-on-the-desktop-layout-that-is-missing-on-mobile">(#2) Content exists on the desktop layout that is missing on mobile</h4>
<p>Allie asks Ted about the “Did you know…” banner that is added on desktop. Ted decided to add this to give the page a bit more visual flair and help with the layout balance.</p>
<p>Allie frowns. “So if someone uses a screen magnifier, they may trigger the mobile version of the page. That content will essentially never be available for that person. That’s also a violation of <a href="https://www.w3.org/WAI/WCAG21/Understanding/resize-text">WCAG Success Criteria 1.4.4: Resize Text (Level AA)</a>. Is that content important?”</p>
<h4 id="3-changes-in-reading-order">(#3) Changes in reading order</h4>
<p>She does a mental outline of the hierarchy and changes in the reading order.</p>
<p>For now, she leaves out sectioning elements, like <code class="highlighter-rouge"><article></code> or <code class="highlighter-rouge"><section></code>. However, she notes that they would be difficult to use properly on the desktop version.</p>
<div style="display: flex; flex-wrap: wrap; margin-bottom: 16px">
<div style="margin: 4px;
padding: 8px;
border: 1px solid #928b80;
border-radius: 4px;">
<strong>Mobile:</strong>
<ul>
<li><strong>img:</strong> Ewoks and et picture</li>
<li>
<strong>h1:</strong> Ewoks and E.T.: A Risky Mix
<ul>
<li><strong>p:</strong> related content</li>
<li>
<strong>h2:</strong> Speeder bikes AND flying bicycles?
<ul>
<li><strong>p:</strong> related content</li>
<li>
<strong>h3:</strong> Speeder bike technology
<ul>
<li><strong>p:</strong> related content</li>
</ul>
</li>
</ul>
</li>
<li><strong>img:</strong> Wookiee picture</li>
<li>
<strong>h2:</strong> But what about Wookiees
<ul>
<li><strong>p:</strong> Wookiee-related content</li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<div style="margin: 4px; padding: 8px; border: 1px solid #928b80; border-radius: 4px;">
<strong>Desktop:</strong>
<ul>
<li>
<strong>h1:</strong> Ewoks and E.T.: A Risky Mix
<ul>
<li><strong>p:</strong> related content</li>
</ul>
</li>
<li>
<strong>img:</strong> Ewoks and E.T. picture
<ul>
<li>
<strong>h2:</strong> But what about Wookiees
<ul>
<li><strong>p: Did you know? ...</strong></li>
<li><strong>p:</strong> related content</li>
</ul>
</li>
<li>
<strong>h2:</strong> Speeder bikes AND flying bicycles?
<ul>
<li><strong>p:</strong> related content</li>
<li>
<strong>h3:</strong> Speeder bike technology
<ul>
<li><strong>p:</strong> related content</li>
</ul>
</li>
</ul>
</li>
<li><strong>img:</strong> Wookiee picture</li>
</ul>
</li>
</ul>
</div>
</div>
<p>Looking at the content this way, she notes that the order of the content changes significantly on desktop.</p>
<p><strong>One of these changes is fine.</strong> The picture related to the “Ewoks and E.T.” content still stays tangential to the content. Whether before or after doesn’t affect meaning or understanding.</p>
<p>However, the sections change reading order, and the Wookiee picture ends up no longer being in a section with its related text.</p>
<p>Allie asks Ted about this. He explains that he wanted the Speeder bikes section to be above the fold. The column design was mostly used to help the image layout.</p>
<p>“Unfortunately,” Allie says, “This will create a reading order that changes meaning on desktop. Not only for screen reader users—but also for desktop users that follow the vertical column flow to read. They will have a disjointed reading experience, where content is read out of order. This is a violation of <a href="https://www.w3.org/WAI/WCAG21/Understanding/meaningful-sequence">WCAG Success Criteria 1.3.2: Meaningful Sequence (Level A)</a>.”</p>
<h3 id="the-pushback">The pushback</h3>
<p>Chadwick is looking a little annoyed at this point. “Allie,” he says, “it’s not our job to question the design, we just implement it.”</p>
<p>Allie shakes her head. “Sorry Chadwick, but it is our job to question a design if we can’t implement it in an accessible way. Not to mention, if the content retains the same order on desktop, there will be other benefits. Namely: performance, SEO, and code maintainability.”</p>
<p>“For example, duplicating content means extra DOM nodes, which is bad for performance and code maintenance. When that content includes heading tags, it can also be bad for SEO.”</p>
<p>Chadwick interjects: “Ok, fine, then we’ll use Javascript to move the content with a resize listener.”</p>
<p>Allie replies, “That covers SEO, but is still affecting reading order, performance, and code maintenance.”</p>
<p>“CSS grid? Rearranging the content in template areas?”</p>
<p>Allie sighs. “Even if you could get that to work, you still would have an issue with the reading order. And any changes in the content would mean reworking the grid.”</p>
<p>At this point, Ted jumps in. “No worries team. I had some of these thoughts myself when working on the design. I’ll take this one back to the drawing board for now, and we can look at it again in the next refinement session.”</p>
<h3 id="the-redesign">The redesign</h3>
<p>Two weeks pass, and Ted has come back with a new design. He’s thrilled to present it.</p>
<figure class="figcaption">
<img class="figcaption__image" style="border: none; border-radius: 0;" src="/static/images/articles/accessibility-plus-content-design/designer-spec_desktop-fixed.png" alt="screen capture of the fixed version of the deskop design, described in caption" />
<figcaption class="figcaption__caption">
<p><strong>Shown above:</strong> The fixed desktop version of the layout.</p>
<p>In this version, the content is in the same general order as the mobile design. There are two rows this time. The first one has the image of the child on the left, and the paragraphs about "Ewoks and ET" and "Speeder bikes" on the right. The next row is centered with a border and contains the Wookiee-related text on the left, with the Wookiee drawing on the right.</p>
</figcaption>
</figure>
<p>“In this new design, I can show the full picture of this fashionable child, including these astounding pink pants. This design should also work better at tablet size than the last one did.”</p>
<p>Allie looks it over, doing a quick mental outline. Now all the content is in the same order/hierarchy, except for the Wookiee picture. But the picture is still tangential to the related content, so that’s fine. She gives a nod of approval. “It looks like you decided to get rid of that extra banner in the Wookiee section?”</p>
<p>“Yeah,” Ted smiles sheepishly, “that never really fit with the purpose of the content. It was just filler - so I took it out.”</p>
<p>Now both developers are happy with the design. Furthermore, Ted has learned to think in a more content-first manner, so his next design will take that into account.</p>
<p>Was it a little awkward at first? Sure. But in the end, these tough conversations result in an improved experience for everyone.</p>
<p>The new design has improved in many ways:</p>
<ul>
<li>It’s more performant and SEO-friendly.</li>
<li>The page has a more natural and readable flow for all users.</li>
<li>The code will be more maintainable, as it won’t need any hacks to work around an inaccessible design. </li>
<li>The developer won’t have to interrupt the designer after they’ve moved on to the next project (for example, to ask for alternative text).</li>
<li>And of course, the page will be accessible: delivering an equal experience for users with disabilities.</li>
</ul>
<p>The bottom line is, you can’t take any design and make it accessible. If the design itself is not accessible, it <strong><em>is</em></strong> your place to speak up as a developer.</p>
<p>Both design and development are more likely to be accessible if they center around the content. If we can all align in that thinking, we can help each other improve.</p>
<h2 id="resources">Resources</h2>
<h3 id="accessibility-basics">Accessibility basics</h3>
<ul>
<li><strong><a href="https://www.w3.org/WAI/">Web Accessibility Initiative (WAI) | W3C</a></strong>: A central hub for learning about accessibility fundamentals. Many of the resources provided below are within this space.</li>
<li><strong><a href="https://www.w3.org/WAI/fundamentals/accessibility-usability-inclusion/">Accessibility, Usability, and Inclusion | WAI | W3C</a></strong>: Understand how accessibility, usability, and inclusive design overlap, and their unique concerns.</li>
<li><strong><a href="https://www.w3.org/WAI/WCAG21/Understanding/">WCAG 2.1 Understanding Docs | WAI | W3C</a></strong>: A deep dive into each of WCAG’s Success Criteria, explaining the intent of the rule, who it benefits, examples, and suggested techniques to meet that criterion.</li>
<li><strong><a href="https://www.w3.org/TR/WCAG21/">WCAG (Web Content Accessibility Guidelines) | WAI | W3C</a></strong>: The official specification, for when you’re ready for all the details.</li>
</ul>
<h3 id="aria---accessible-rich-internet-applications">ARIA - Accessible Rich Internet Applications</h3>
<ul>
<li><strong><a href="https://www.w3.org/TR/using-aria/">Using ARIA | WAI | W3C</a></strong>: 5 really important rules to know before using ARIA, plus other tips and best practices.</li>
<li><strong><a href="https://www.w3.org/WAI/ARIA/apg/">ARIA Authoring Practices Guide (APG) | WAI | W3C</a></strong>: Component design patterns and examples, best practices, and much more.</li>
<li><strong><a href="https://www.w3.org/TR/wai-aria-1.1/">Accessible Rich Internet Applications (WAI-ARIA) 1.1 | WAI | W3C</a></strong>: The official ARIA spec. This is a lot to dive into without some basic background first, I suggest starting with the links above.</li>
</ul>
<h3 id="semantic-html">Semantic HTML</h3>
<ul>
<li><strong><a href="https://developer.mozilla.org/en-US/docs/Learn/Accessibility/HTML">HTML: A good basis for accessibility | MDN web docs</a></strong>: The basics of semantic HTML and the benefits it offers.</li>
<li><strong><a href="https://24ways.org/2017/accessibility-through-semantic-html/">Accessibility Through Semantic HTML | Laura Kalbag, 24 Ways</a></strong>: A great summary of the benefits of semantic HTML, and how to test your work.</li>
</ul>
<h3 id="content-design">Content Design</h3>
<ul>
<li><strong><a href="https://contentdesign.london/blog/what-is-content-design">What is content design? | Content Design London</a></strong>: A great intro for understanding content design and how it answers users’ needs.</li>
<li><strong><a href="https://www.writingisdesigning.com/">Writing is Designing | Michael J. Metts and Andy Welfle (book)</a></strong>: How to approach writing for the web as a form of design.</li>
<li><strong><a href="https://www.prosekiln.com/blog/2020/11/17/the-massive-list-of-content-design-amp-ux-writing-resources">The Massive List of Content Design & UX Writing Resources | Prose Kiln</a></strong>: A list of books, articles, websites, courses, and much more.</li>
</ul>
<h3 id="tutorials-and-resources-for-beginners">Tutorials and resources for beginners</h3>
<ul>
<li><strong><a href="https://www.w3.org/WAI/courses/foundations-course/">Digital Accessibility Foundations Free Online Course | W3C WAI</a></strong>: An intro accessibility course for developers, designers, content authors, and anyone else who wants to learn the basics.</li>
<li><strong><a href="https://web.dev/learn/accessibility/">Learn Accessibility | web.dev</a></strong>: A great overview of the basics, and a good reference.</li>
<li><strong><a href="https://testingaccessibility.com/buy">Testing Accessibility Courses (paid course) | Marcy Sutton</a></strong>: A well-known accessibility advocate, Marcy offers three levels of courses with a hands-on, simplified approach to testing the accessibility of your web pages.</li>
<li><strong><a href="https://www.youtube.com/watch?v=pOUeNj7Pnc4">Introduction to Screen Readers (video) | YouTube, Tim Harshbarger (A11yTalks - February 2021)</a></strong>: Tim teaches about screen reader basics and how testing with screen readers is different from the way they’re actually used.</li>
</ul>
<h3 id="tools-and-reference">Tools and reference</h3>
<p>There are far too many tools to list all the good ones. This list is slimmed down to some of those tools I use on a regular basis. Start with a few and see how they work for you.</p>
<ul>
<li><strong><a href="https://www.w3.org/WAI/WCAG21/quickref/">How to Meet WCAG (Quick Reference) | WAI | W3C</a></strong>: A filterable quick reference list of all the WCAG criteria. A great way to find criteria by role, level, or subject (for example, “audio”).</li>
<li><strong><a href="https://a11ysupport.io/">Accessibility Support (allysupport.io)</a></strong>: Check elements, attributes, and more to find details on feature support. While not always 100% up to date, it’s still one of the best tools for this task.</li>
<li><strong><a href="https://accessibilityinsights.io/">Accessibility Insights</a></strong>: A testing tool created by Microsoft, available for Web (Chrome and Edge), Android, and Windows. It offers the ability to scan your page for issues using the axe-core library, as well as tools that support manual testing.</li>
<li><strong><a href="https://dequeuniversity.com/screenreaders/">Screen Reader Keyboard Shortcuts and Gestures | Deque University</a></strong>: Great shortcut guides for lots of different screen readers, in web or pdf versions.</li>
<li><strong><a href="https://www.tpgi.com/color-contrast-checker/">Color Contrast Checker | TPGi</a></strong>: An installed application for Windows or macOS, it can test color contrast on live sites or design mock-ups with a simple eyedropper tool.</li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><strong><a href="https://www.mdpi.com/2227-9709/7/1/8">Accessibility in Web Development Courses: A Case Study | MDPI</a></strong></li>
</ul>Harmony Hameshhames@rei.comA silly story that helps illustrate the benefits of understanding both accessibility and content design.The front-end build tool renaissance at REI2023-02-02T00:00:00+00:002023-02-02T00:00:00+00:00https://engineering.rei.com/frontend/front-end-build-tool-renaissance<p>The Front-end Build System (FEBS) is REI’s internally developed set of tools that helps engineers build the assets that are eventually downloaded by customers visiting REI’s online properties. FEBS has evolved over the years to support an ever-changing development landscape.</p>
<p><img src="/static/images/articles/febs/rei-febs-logo.png" alt="REI FEBS logo" class="article-image-full-width" /></p>
<p>The first generation of FEBS was built directly in our legacy monolithic code base, a series of scripts that leveraged some programmable APIs from libraries like <code class="highlighter-rouge">gulp</code> and <code class="highlighter-rouge">browserify</code>. This code continues to serve any remaining applications living in that monolithic space.</p>
<p>The Co-op moved toward a more decentralized approach by introducing microservice architecture with the development of the <a href="https://engineering.rei.com/devops/how-we-built-a-microservices-platform.html">Alpine platform</a>. With this came a new generational iteration of <a href="https://engineering.rei.com/frontend/the-rei-front-end-build-system.html">FEBS</a> that was decoupled from any individual implementation. FEBS 2 lives in REI’s internal NPM registry as <code class="highlighter-rouge">@rei/febs</code> with several other supporting packages. It is declared as a dependency of an Alpine microsite.</p>
<p>Under the hood, <code class="highlighter-rouge">@rei/febs</code> is extending <code class="highlighter-rouge">webpack@4</code> to bundle web application code and <code class="highlighter-rouge">rollup@2</code> to bundle Vue 2 components and vanilla JavaScript libraries.</p>
<h2 id="unforeseen-consequences">Unforeseen consequences</h2>
<p>Our team maintains a microsite that contains supported patterns, technologies, and most importantly, an end-to-end functional example used as a reference by other product teams. We expected <code class="highlighter-rouge">@rei/febs</code> to canonicalize the way teams built their front-end assets and thereby reduce any individual deviation from supported patterns, theoretically improving the interoperability ecosystem.</p>
<p>For most cases, <code class="highlighter-rouge">@rei/febs</code> did its job. However, as time went on, feature requests continued to pour in to accommodate product teams with unique business requirements. Additionally, the underlying dependencies supporting <code class="highlighter-rouge">@rei/febs</code> continued to change, forcing us to manually reconcile them within our packages and within individual microsites.</p>
<p>The maintenance of iterating on <code class="highlighter-rouge">@rei/febs</code> and troubleshooting microsite build issues began to consume our daily lives. Worse was that <code class="highlighter-rouge">@rei/febs</code> was becoming bloated with logic for many unforeseen use cases.</p>
<h2 id="inherit-apparent">Inherit apparent</h2>
<p>Perhaps you spotted the <em>inherent</em> problem with <code class="highlighter-rouge">@rei/febs</code>? It’s an inheritance model. <code class="highlighter-rouge">@rei/febs</code> extends several build tool technologies including <code class="highlighter-rouge">webpack</code> and <code class="highlighter-rouge">rollup</code>. This means that <code class="highlighter-rouge">@rei/febs</code> <strong>is</strong> <code class="highlighter-rouge">webpack</code>. <code class="highlighter-rouge">@rei/febs</code> <strong>is</strong> <code class="highlighter-rouge">rollup</code>. So, what’s the issue? Inheritance models are brittle by nature (don’t @ me). Any change request can require a massive overhaul of the implementation. And the more you customize, the harder it becomes to maintain until it becomes an unrecognizable, enormous blob like in <em>Akira</em>.</p>
<h2 id="this-is-the-way">This is the way</h2>
<p>Eventually, we hit a wall when Vue 3 arrived on the scene. We built a POC that brought Vue 3 support to <code class="highlighter-rouge">@rei/febs</code>. We discovered that the Rollup plugin used to compile Vue 3 SFCs for our public design system, <a href="https://rei.github.io/rei-cedar-docs/">Cedar</a>, made tree-shaking incompatible with <code class="highlighter-rouge">webpack@4</code>, resulting in bloated application bundles.</p>
<p>Even though we had a functional POC of our reference microsite using Vue 3, we were unsatisfied with the low-level work we had to do to get there. We decided to experiment with a build toolchain overhaul that dispensed with our implementations in favor of using <a href="https://vitejs.dev/">Vite</a> to bundle our application. We quickly cobbled together a working solution. We liked the approach so much that we decided to move forward with it.</p>
<h2 id="enter-vite">Enter Vite</h2>
<p><a href="https://vitejs.dev/">Vite</a> is a UI-framework agnostic build tool. It supports React, Svelte, Vue, and others. It’s a bundler … of bundlers. Ironically, it’s a wrapper that swaps bundlers in and out for different purposes. But the key point here is that it’s a wrapper maintained by an open-source community <strong>instead</strong> of us. Under the hood, it uses <code class="highlighter-rouge">rollup</code> and <code class="highlighter-rouge">esbuild</code>.</p>
<p>Vite happens to have first-class support for Vue, which makes sense, as it was built by the creator of Vue. REI is a Vue shop. Our design system, <a href="https://rei.github.io/rei-cedar-docs/">Cedar</a>, is a Vue component library. Cedar now uses Vite to bundle its code. We’ll use Vite to bundle our customer-facing Vue 3 applications. One tool to rule them all.</p>
<p>Besides all that, Vite is pretty sweet. It has a “no-bundler” approach to running its development server. It doesn’t need to build your application before it’s served.</p>
<blockquote>
<p>Vite serves source code over native ESM. This is essentially letting the browser take over part of the job of a bundler: Vite only needs to transform and serve source code on demand, as the browser requests it. Code behind conditional dynamic imports is only processed if actually used on the current screen.</p>
</blockquote>
<h2 id="so-what-is-febs-now">So, what is FEBS now?</h2>
<p>FEBS 3 is the “Front-end Build System”, but that system isn’t an explicit dependency.</p>
<p>Rather, it’s a curated collection of open-source tools that are declared as dependencies of Alpine microsites and NPM packages used to bundle assets. Specifically Vite and Vitest. These tools are supplemented by REI-built configurations, plugins, and patterns to support REI-specific needs.</p>
<p>This has alleviated us from low-level dependency management, makes future migrations and tool swaps easier, and allows engineers maximum flexibility with their application builds while always having a functional example to reference. Engineers can also lean directly on the open-source community for any issues they may have with a particular build tool.</p>
<h2 id="lessons-learned">Lessons learned</h2>
<p>Reflecting on the evolution of the build toolchain at REI, the intentions were good. Create a tool that would normalize the way our teams built front-end assets and prevent deviation from supported patterns.</p>
<p>What we learned was that we didn’t need to be restrictive in the form of an explicit tool like <code class="highlighter-rouge">@rei/febs</code>. It’s not like people couldn’t get around it anyway. Instead, we simply provide sanctioned patterns and people use them. The benefits of using the raw build tool greatly outweigh a customization that extends the tool directly.</p>Kurt Medleykmedley@rei.comThe Front-end Build System (FEBS) is REI’s internally developed set of tools that helps engineers build the assets that are eventually downloaded by customers visiting REI’s online properties. FEBS has evolved over the years to support an ever-changing development landscape.How We Made Managing CSP Less Annoying2023-01-23T00:00:00+00:002023-01-23T00:00:00+00:00https://engineering.rei.com/security/making-csp-less-annoying<p>At REI, the Cybersecurity Engineering teams constantly work hand in hand with our application and infrastructure teams. The outdoor analogy that sometimes comes to mind is that the Cybersecurity Engineering team are the belayers, and the application and infrastructure teams are the climbers. When it comes to securing our applications at REI, we want to ensure that developers can continue to climb to new heights, with our Cybersecurity engineers guiding and assisting along the way.</p>
<h1 id="the-journey-to-content-security-policy-csp">The Journey to Content Security Policy (CSP)</h1>
<p>In the beginning of 2022, both the Cybersecurity Engineering team and the Site Reliability Engineering (SRE) team saw that CSP could help solve several of our questions:</p>
<ul>
<li>Is there an inventory of 3rd party Javascript that is used on REI.com?</li>
<li>How do we control what 3rd party Javascript gets loaded on the client browser when someone visits REI.com?</li>
<li>How do we ensure that when Javascript loads on a client browser, it can’t communicate with inappropriate or nefarious domains?</li>
<li>How can we better prevent potential data skimming attacks?</li>
</ul>
<p><br /></p>
<p>While there are other security controls that can help in detecting a compromise of our first party Javascript, the CSP provides an additional defense-in-depth layer, preventing software supply chain attacks where a possible script tag may be compromised.</p>
<h1 id="how-does-csp-work">How Does CSP Work?</h1>
<p>Content Security Policy, or CSP for short, declares to the client browser a set of approved sources that can run on a web page. Another way to think about CSP is that it’s a way to make an allow-list of things that your client browser has permissions to load and run.</p>
<p>CSP is implemented using a <code class="highlighter-rouge">Content-Security-Policy</code> HTTP response header, and it can help answer questions like:</p>
<ul>
<li>Where can I get images from?</li>
<li>Where can I get stylesheets from?</li>
<li>Where can I get JavaScript from?</li>
<li>Can JavaScript run inline in the page?</li>
<li>Can the page be embedded in another site’s frame?</li>
</ul>
<p><br /></p>
<p>By enabling CSP, we are being explicit on where our website can embed and load content from. When you load a web page, there’s other things that get embedded in that page, such as stylesheets, JavaScript, images, media, so on a so forth. Very often, these assets come from various other locations – for example, media from YouTube.</p>
<p>CSP is about security, and one of the main reasons why you would want to implement CSP is that it’s meant to detect, reduce, and/or eliminate the risk of threats such as cross-site scripting, packet sniffing, and other data injection attacks.</p>
<p>At a high level, when a web browser goes to www.rei.com, they receive a response from our web application that includes a response header of <code class="highlighter-rouge">Content-Security-Policy</code> along with a value of directives. These directives have many functions and basically gives instructions to the client browser on what can be loaded. If something is not specified in a directive, it will be blocked by the client browser.</p>
<p><img src="/static/images/articles/making-csp-less-annoying/csp_flow.png" alt="flow of how CSP works" class="article-image-full-width" /></p>
<p>There are a lot more details into exactly what each directive does and, to make things more complicated, some browsers may interpret or respect directives differently. As we were writing the policy, we found that the <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP">Mozilla Developer Network’s CSP page</a> very helpful.</p>
<h1 id="things-we-learned-along-the-way">Things We Learned Along the Way</h1>
<p>We knew early on that we would have to an exhaustive discovery of all the third party Javascript being loaded on REI.com and verifying internally that someone had a relationship with that third party vendor. To gather all that data we relied on using the <code class="highlighter-rouge">Content-Security-Policy-Report-Only</code> header at first. This header basically places the policy in a “monitor” mode and will not block Javascript being loaded. However, it will still send the results of what it would have blocked. This helped us in continually iterating over the policy until we were confident that we would not block any legitimate scripts.</p>
<p>We also needed a way to collect and analyze the data. By using the <code class="highlighter-rouge">report-uri</code> directive we were able to send any CSP violations to a collection endpoint for further analysis. As you can imagine, there can be a lot of reports coming from client browsers going to our site so having a way to organize and visualize the data was really important. We ultimately accomplished this by using a third party vendor that specialized in organizing all the CSP data for us.</p>
<p>After all that, we finally had a workable CSP to push into our production environment. The next step was to find a process to maintain our CSP.</p>
<h1 id="csp-challenges">CSP Challenges</h1>
<p>One of the challenges with how we did CSP initially was that our policy directives were not in a formal source control system. This meant that whenever we had to make any changes to our CSP directives, we had to do the following:</p>
<ol>
<li>Copy the one-line CSP values directly from our CDN configuration</li>
<li>Paste that line onto a text editor</li>
<li>Make the changes while still maintaining the single-line layout</li>
<li>Paste the modified line back into the CDN configuration</li>
<li>Hope that we didn’t break CSP</li>
</ol>
<h1 id="solutions">Solutions</h1>
<p>To introduce source control, we simply copied and pasted the single-line CSP header from our configuration and into a text file that we checked into git. This solution worked okay at first, but after a few times having to update the CSP we quickly understood that we need a better process — specifically around managing the single-lined CSP header.</p>
<p><img src="/static/images/articles/making-csp-less-annoying/mark_hamill.png" alt="Actor Mark Hamill in YAML form" class="article-image-full-width" /></p>
<p><em>So, we converted the one-line into YAML!</em> <a href="https://yaml.org/">YAML</a> is a data serialization language, and we chose that over other alternatives mainly due to YAML’s readability factor. The format of a YAML file is more human-readable and easier to manage than something like JSON.</p>
<p>Let’s take this CSP header for example:</p>
<p><br /></p>
<p><code class="highlighter-rouge">frame-ancestors 'self' https://www.coolwebsite.com; default-src 'self' https://*.rei.com; script-src 'self' 'unsafe-eval' blob: https://www.coolwebsite.com https://*.another-website.com; img-src data: *; report-uri https://cspissues.com</code></p>
<p><br /></p>
<p>After giving it the YAML treatment, the line above would look like this:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">frame-ancestors</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">self'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">https://www.coolwebsite.com'</span>
<span class="na">default-src</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">self'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">https://*.rei.com'</span>
<span class="na">script-src</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">self'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">unsafe-eval'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">blob:'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">https://www.coolwebsite.com'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">https://*.another-website.com'</span>
<span class="na">img-src</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">data:'</span>
<span class="pi">-</span> <span class="s1">'</span><span class="s">*'</span>
</code></pre></div></div>
<p>This version is easier on the eyes don’t you think?
We converted our gigantic CSP header into YAML and developed a script that converts the YAML file to the single-lined CSP header, and Voila! Our new process with these improvements now looks like this:</p>
<ol>
<li>Make a change to the CSP YAML file and create a pull request</li>
<li>Once approved, merge the pull request and generate the single-lined CSP header value using the script</li>
<li>Replace current CSP header defined in our CDN configuration with the newly created single-lined CSP header value</li>
<li>Go on a hike to celebrate!</li>
</ol>
<h1 id="conclusion">Conclusion</h1>
<p>We now have a more readable, source control friendly, easier to manage version of CSP that even non-technical personnel can make changes if they wish. This collaboration between the Cybersecurity Engineering team and SRE is a perfect example of people from different expertise coming together to do more than they can on their own, fufulling one of REI’s core values: “We go further together!”</p>Dan Ngodngo@rei.comAt REI, the Cybersecurity Engineering teams constantly work hand in hand with our application and infrastructure teams. The outdoor analogy that sometimes comes to mind is that the Cybersecurity Engineering team are the belayers, and the application and infrastructure teams are the climbers. When it comes to securing our applications at REI, we want to ensure that developers can continue to climb to new heights, with our Cybersecurity engineers guiding and assisting along the way.Catching the Wheel2022-11-26T00:00:00+00:002022-11-26T00:00:00+00:00https://engineering.rei.com/web-performance/catch-the-wheel<p>Days before the holiday sales change freeze, the web performance and analytics teams deployed a change that drastically improved front-end performance. A new configuration on a third-party script led to a 60% improvement to <a href="https://web.dev/fcp">First Contentful Paint (FCP)</a>. This rocketed us to the top of our industry performance leader board. Most importantly, our real user data met Google’s Core Web Vitals goals: A strong indication of a great customer experience.</p>
<p>This change took months of deliberation. We collaborated across teams and with our A/B testing vendor. This journey entails good times, bad times and some humbling realizations. You won’t find a clearly defined list of tips for overcoming third-party vendor performance woes. This is a story with events I hope help guide you in tackling large, seemingly insurmountable problems like we faced.</p>
<h2 id="if-a-tree-falls-in-the-forest">If a Tree Falls in the Forest…</h2>
<p>In the field, REI leverages Google’s <a href="https://web.dev/vitals">Core Web Vitals (CWV)</a>. These quality signals uncover real users’ experience on rei.com with metrics like FCP and <a href="https://web.dev/lcp">Largest Contentful Paint (LCP)</a>. These metrics signify critical points in a web page’s loading process. Without CWV, loading issues fly under the radar. After all, software engineers typically use powerful laptops on stable connections.</p>
<p>Our CWV scores hovered between “needs improvement” and poor. We had to get to the bottom of it.</p>
<h2 id="the-state-of-web-perf-in-the-industry">The State of Web Perf in the Industry</h2>
<p>To identify issues, the web performance team runs lab tests with emulated affordable devices on slower connections. <strong>Everyone should do this.</strong> Making a website fast on an affordable device makes it fast for everyone. Throttled lab tests make issues more pronounced and easier to prioritize, too–win-win.</p>
<p><img src="/static/images/articles/catch-the-wheel/baseline.png" alt="Timeline showing the loading process of rei.com. The page doesn't render anything until 8 seconds." class="article-image-full-width" /></p>
<p>The homepage on a mobile device with 3G doesn’t look great. That’s a pretty slow configuration, though. Almost every site should run that slow, right? Let’s take a look at some other sites.</p>
<p><img src="/static/images/articles/catch-the-wheel/category-before-blur.png" alt="Loading timelines of REI.com and other industry websites. REI loads the slowest compared to the rest." class="article-image-full-width" /></p>
<p>Ouch! REI’s search page sits dead last compared to other search pages. If you’re an REI engineer reading this, you’re probably shocked. We build our pages with <a href="https://www.patterns.dev/posts/server-side-rendering/">Server Side Rendering (SSR)</a> and efficiently cache them at the edge. Content should render the instant a browser receives HTML. If we’re doing all of this work, something external must be at fault.</p>
<p>In bicycle racing, it’s key for cyclists to “catch the wheel” of their competitors. Sitting in your competitors’ draft removes a great deal of wind resistance and allows you to pedal softer. While in the draft you can recover to prepare yourself for the sprint to the finish. The web performance team called this effort “catch the wheel” to drive the point home. If we remove this performance roadblock, we’ll be in the front of the pack and ready to sprint for first place.</p>
<h2 id="identifying-the-problem">Identifying the Problem</h2>
<p>If your FCP and LCP warning lights go off, it’s time to evaluate your critical rendering path. Something’s probably gumming it up. When we first investigated the issue, we had 4 render blocking assets:</p>
<ol>
<li>A small security related script</li>
<li>CSS for a global component (a problem for later)</li>
<li>A tag manager</li>
<li>A client-side A/B testing script</li>
</ol>
<p> </p>
<p>One of these must make a huge impact. The quick and dirty solution is to run tests with each critical asset blocked, one by one. Compare the tests and see which asset makes the biggest impact. When we got to our A/B testing script, things cleared up.</p>
<p><img src="/static/images/articles/catch-the-wheel/deferred%20adobe%20target.png" alt="Timeline showing REI.com starts loading at 3.5 seconds after blocking the A B testing script." class="article-image-full-width" /></p>
<p>An FCP change from 8 second to 3.5 seconds is a massive improvement. Why does it make such a huge impact?</p>
<h2 id="an-aside-on-anti-flicker">An aside on “anti-flicker”</h2>
<p>Client-side A/B solutions attempt to resolve an issue called “flicker” in two ways:</p>
<ol>
<li>Make their script render-blocking</li>
<li>Run an “anti-flicker” snippet that hides the page with opacity: 0 until targeted content arrives</li>
</ol>
<p>“Flicker” denotes an experience where the user sees default content that swaps with targeted content. To be fair, that’s a pretty jarring experience. Flicker makes pages inaccessible, too. A unexpected content swap messes with cognitive load and pulls the rug out from under screen readers.</p>
<p>Here’s an extreme example where flicker happens above-the-fold on a hero element:</p>
<video controls="" width="100%" title="Example of Default Content Swapping with Targeted Content">
<source src="/static/images/articles/catch-the-wheel/flicker-example.mov" />
</video>
<p>On the flip side, anti-flicker destroys page speed. Our lab tests indicate a 4.5 second increase to FCP on mobile devices. We’re trading bad UX for bad UX. If we have a bad testing experience and bad performance, where can we make a comprise? Here comes the humbling realization part.</p>
<h2 id="unblock-the-rendering-path">Unblock the Rendering Path</h2>
<p>Since I joined REI in March, the performance solution to the A/B problem seemed obvious: make the script non-blocking and allow flicker(disable anti-flicker). A performance consultant suggested something similar. They kept it simple and only recommended turning off anti-flicker. After some testing only disabling anti-flicker unfortunately didn’t change much. This diagram helps explain why:</p>
<p><img src="/static/images/articles/catch-the-wheel/baseline-ab-diagram.png" alt="" class="article-image-full-width" /></p>
<p>The real crux of the issue lies in the render blocking nature of the script. Everything screeches to a halt until that script gets dealt with. Let’s see how the page would load if we add “defer” to that script:</p>
<p><img src="/static/images/articles/catch-the-wheel/deferred-ab-diagram.png" alt="" class="article-image-full-width" /></p>
<p>With the script deferred, the page can render as soon as possible. The browser downloads the script in the background and waits to run it after the HTML gets parsed.</p>
<p>Deferring scripts is a great web performance technique. Use it wisely, though, because it changes the execution order of scripts on the page. The A/B script lives in the <code class="highlighter-rouge"><head /></code>. With <code class="highlighter-rouge">defer</code> present, the browser will get to it after scripts at the end of the <code class="highlighter-rouge"><body /></code> run. When a script at the end of the <code class="highlighter-rouge"><body /></code> also has <code class="highlighter-rouge">defer</code>, the execution order is preserved. The browser will queue them up based on their position in the HTML.</p>
<p>Here’s a high level defer diagram:
<img src="/static/images/articles/catch-the-wheel/defer-diagram.png" alt="Bar diagram showing how a script blocks rendering until it downloads, parses, compiles and executes. Another bar diagram below shows how a deferred script downloads in the background and waits to parse compile and execute after the HTML is parsed." class="article-image-full-width" /></p>
<p>This proposal looks great for the web performance team, but what about the analytics team? When we defer the script and disable anti-flicker, we guarantee flicker. This boost to performance degrades the testing experience. REI or any company won’t accept this compromise. Unless…</p>
<h2 id="i-made-a-massive-oversight">I Made A Massive Oversight</h2>
<p>The problem appeared impossible to solve. Changing vendors to <em>maybe</em> have a faster client-side A/B solution costs too much. Improving performance, but making our UX jarring isn’t a great trade. Our proposal seemed like the only solution, and no one would accept it unless it preserved or improved the testing experience. After some cross team face-to-face brainstorming sessions we realized something huge: <strong>flicker already happens in production</strong>.</p>
<p>For months I assumed the anti-flicker mechanism worked. In all of my performance tests, I hadn’t witnessed flicker. Our consultant even suggested accepting flicker as if we had that solved the whole time. It turns out a race condition occurs between our A/B script and our front-end framework, Vue. The A/B script un-hides the page once it receives targeted content, but that’s too soon. That content needs to get processed by Vue before rendering. This race condition caused flicker.</p>
<p><img src="/static/images/articles/catch-the-wheel/baseline-flicker-diagram.png" alt="Diagram revealing a delay in rendering targeted content because the content must get added to the virtual dom by Vue." class="article-image-full-width" /></p>
<p>Flicker was ingrained in the testing culture at REI. The marketing team knew they should only test elements below-the-fold. They avoid large, above-the-fold tests, like the one in the video. The testing team worked around the problem and I was none the wiser. To be fair, I recall the analytics team mentioning flicker impacted tests, but I assumed it was rare. Never make assumptions! This was a huge learning experience for me.</p>
<p>We met with the vendor a few times to make sure deferring the script wouldn’t break anything. We found out some other companies already defer the script and accept flicker. Unfortunately that wasn’t documented anywhere. Face to face conversations are hard, but again proved themselves to be crucial in this whole process.</p>
<h2 id="rollout">Rollout</h2>
<p>Now that we confirmed the testing experience won’t change and got vendor approval, it was time to roll out the performance improvement–with one week left until the code freeze. The analytics team leveraged our in-house feature toggle solution to make adding “defer” and disabling anti-flicker a breeze. This configuration based approach gave us a fast fallback if something went wrong. The web performance team reached out to every site owner within rei.com to make sure their site could accept a deferred A/B script. 10 PRs later, we were ready to roll.</p>
<h2 id="results">Results</h2>
<p>We rolled out the change at the end of October. The results were astounding. I’ve shown a lot of lab test data, so let’s look at the real user impact.</p>
<h3 id="fcp-improvements-to-the-search-page">FCP Improvements to the Search Page</h3>
<p><img src="/static/images/articles/catch-the-wheel/RUM%20Search%20FCP.png" alt="Line graph with lines representing the 95th and 75th percentile of users' FCP scores. The day the change rolled out, the line drops significantly." class="article-image-full-width" /></p>
<h3 id="lcp-improvements-to-product-pages">LCP Improvements to Product Pages</h3>
<p><img src="/static/images/articles/catch-the-wheel/RUM%20PDP%20LCP.png" alt="Line graph with lines representing the 95th and 75th percentile of users' LCP scores. The day the change rolled out, the line drops significantly." class="article-image-full-width" /></p>
<p>This change brought our average FCP to Google’s recommended zone. A lot of our LCP scores now meet recommendations, too. Although there are some pages that need more work. This performance improvement uncovered a lot of issues hidden by the render-blocking script. Now that the page loads faster, we can eke out better LCP scores and tackle new issues with <a href="https://web.dev/cls">CLS</a>, <a href="https://web.dev/tbt">TBT</a> and more.</p>
<p>Back to the lab tests, we compiled a list of the improvements to LCP and FCP scores:</p>
<h3 id="fcp">FCP</h3>
<table>
<thead>
<tr>
<th>Page</th>
<th>FCP Before (seconds)</th>
<th>FCP After (seconds)</th>
<th>Delta (seconds)</th>
</tr>
</thead>
<tbody>
<tr>
<td>Homepage</td>
<td>5.6</td>
<td>2.01</td>
<td>-3.6</td>
</tr>
<tr>
<td>Search pages</td>
<td>7.06</td>
<td>1.9</td>
<td>-5.2</td>
</tr>
<tr>
<td>Brand (custom landing) pages</td>
<td>5.51</td>
<td>2.15</td>
<td>-3.7</td>
</tr>
<tr>
<td>PDP</td>
<td>5.52</td>
<td>1.92</td>
<td>-3.6</td>
</tr>
<tr>
<td>Outlet PDP</td>
<td>5.44</td>
<td>1.94</td>
<td>-3.6</td>
</tr>
<tr>
<td>Expert Advice home</td>
<td>5.4</td>
<td>1.7</td>
<td>-3.7</td>
</tr>
<tr>
<td>Expert Advice articles</td>
<td>5.22</td>
<td>2.67</td>
<td>-2.6</td>
</tr>
<tr>
<td>Adventures home</td>
<td>5.93</td>
<td>2.27</td>
<td>-3.7</td>
</tr>
<tr>
<td>Adventures trip pages</td>
<td>5.28</td>
<td>2.25</td>
<td>-3</td>
</tr>
<tr>
<td>Classes & Events home</td>
<td>6.55</td>
<td>1.89</td>
<td>-4.7</td>
</tr>
<tr>
<td>Classes & Events detail pages</td>
<td>5.83</td>
<td>2.1</td>
<td>-3.7</td>
</tr>
<tr>
<td>Retail store pages</td>
<td>5.1</td>
<td>1.95</td>
<td>-3.2</td>
</tr>
<tr>
<td>Newsroom</td>
<td>5.86</td>
<td>2.19</td>
<td>-3.7</td>
</tr>
</tbody>
</table>
<h3 id="lcp">LCP</h3>
<table>
<thead>
<tr>
<th>Page</th>
<th>LCP Before (seconds)</th>
<th>LCP After (seconds)</th>
<th>Delta (seconds)</th>
</tr>
</thead>
<tbody>
<tr>
<td>Homepage</td>
<td>5.6</td>
<td>2.94</td>
<td>-2.7</td>
</tr>
<tr>
<td>Search pages</td>
<td>6.45</td>
<td>3.67</td>
<td>-2.8</td>
</tr>
<tr>
<td>Brand (custom landing) pages</td>
<td>5.6</td>
<td>3.13</td>
<td>-2.5</td>
</tr>
<tr>
<td>PDP</td>
<td>5.68</td>
<td>2.61</td>
<td>-3.1</td>
</tr>
<tr>
<td>Outlet PDP</td>
<td>5.44</td>
<td>2.36</td>
<td>-3.1</td>
</tr>
<tr>
<td>Expert Advice home</td>
<td>5.4</td>
<td>3.9</td>
<td>-1.5</td>
</tr>
<tr>
<td>Expert Advice articles</td>
<td>5.22</td>
<td>3.78</td>
<td>-1.4</td>
</tr>
<tr>
<td>Adventures home</td>
<td>5.93</td>
<td>2.98</td>
<td>-3</td>
</tr>
<tr>
<td>Adventures trip pages</td>
<td>5.34</td>
<td>4.38</td>
<td>-1</td>
</tr>
<tr>
<td>Classes & Events home</td>
<td>5.71</td>
<td>4.47</td>
<td>-1.2</td>
</tr>
<tr>
<td>Classes & Events detail pages</td>
<td>5.83</td>
<td>2.73</td>
<td>-3.1</td>
</tr>
<tr>
<td>Retail store pages</td>
<td>5.34</td>
<td>2.83</td>
<td>-2.5</td>
</tr>
<tr>
<td>Newsroom</td>
<td>5.86</td>
<td>3.02</td>
<td>-2.8</td>
</tr>
</tbody>
</table>
<p>And lastly, let’s see how REI stacks up against other industry websites:</p>
<p><img src="/static/images/articles/catch-the-wheel/category-after-blur.png" alt="Timeline comparison between REI's search page and other search pages. REI ties for first place with the leader" class="article-image-full-width" /></p>
<p>We caught the wheel and now sit at the front of the pack! In some cases we’re in the lead and in others we’re in the perfect position to lay down power and sprint for first place.</p>
<p><img src="/static/images/articles/catch-the-wheel/home-after-blur.png" alt="Timeline comparison between REI's home page and other home pages. REI sits neck and neck with second place, below the top two websites" class="article-image-full-width" /></p>
<h2 id="the-future-of-ab-testing-at-rei">The Future of A/B Testing at REI</h2>
<p><em>Wait… Aren’t you forgetting something?</em></p>
<p>Our solution didn’t solve flicker. We noticed it’s always happened and left it where it is. This experience continues to limit the marketing teams in how they design and perform tests. We can do better!</p>
<p>Our vendor suggested we implement their element hiding solution. We can selectively hide elements until we receive targeted content then render them. We can take it a step further and build a loading treatment for these elements. Our design system already had this in their backlog and just needed the right use case.</p>
<p>This solution will enable tests above the fold and extend marketing’s capabilities. We’ll keep the script deferred, too. The page will continue to render as fast as it does today.</p>
<h3 id="one-last-compromise">One Last Compromise</h3>
<p>An element hiding solution requires a bit of manual work. Developers or testers need to mark an element as the one getting tested so it can have a loading state. This is a classic case of user experience (UX) over developer experience (DX). The modern web has a problem with favoring DX over UX. That’s why JavaScript size has ballooned in recent years. <a href="https://httparchive.org/reports/state-of-javascript?start=2015_08_15&end=latest&view=list">(HTTP Archive Average JS Size Over Time)</a> As web developers, we need to put the user first. The web is a beautiful thing. Let’s not squander it.</p>
<h2 id="kudos-and-shoutouts">Kudos and Shoutouts</h2>
<p>This was hardly a solo effort. This massive leap in performance wouldn’t have been possible without everyone involved at the co-op. Erik Luchauer and Marina Robbins handled the meetings, work prioritization and communications. From the analytics team, Brian Mendez and Evan Chang were right there with us through the end. Brian set up the rollout process so it could go as smoothly as it did and Evan kept us focused and advocated for web performance. Kat Valdre brought us front and center to our vendor so we could hash out details and get that critical green light. Every site team at REI promptly brought in our pull requests and chipped in if something got left out along the way. And thank you to the design system team for bringing in the progressive loading work so quickly!</p>
<p> </p>Casey Carrollcacarro@rei.comDays before the holiday sales change freeze, the web performance and analytics teams deployed a change that drastically improved front-end performance. A new configuration on a third-party script led to a 60% improvement to First Contentful Paint (FCP). This rocketed us to the top of our industry performance leader board. Most importantly, our real user data met Google’s Core Web Vitals goals: A strong indication of a great customer experience.