Unit testing • selenider

selenider is compatible with unit testing frameworks like testthat and shinytest2. In this vignette, we will explore how to write unit tests with selenider, and we will briefly describe how to automate your tests using Github Actions.

library(selenider)
library(testthat)

Using selenider with testthat

Tests contained within testthat::test_that() are self-contained, having no impact on other tests. selenider is no exception: selenider sessions, when created inside a testthat::test_that() block, will be closed automatically when the test finishes running.

Remember, as always, to use the .env argument when wrapping selenider_session() in another function.

elem_expect() also has additional features inside testhat::test_that(). When it succeeds, it will call testthat::succeed(), and when it fails, it will use testthat::fail() instead of throwing an error. This allows tests to continue running even if elem_expect() fails.

test_that("My test", {
  # session will be opened here...
  open_url("https://www.r-project.org/")

  s(".random-class") |>
    elem_expect(is_present)
}) # and closed here!
#> -- Failure: My test ------------------------------------------------------------
#> Condition failed after waiting for 4 seconds:
#> `is_present`
#> i `x` is not present.
#> 
#> Where `x` is:
#> A selenider element selecting:
#> The first element with css selector ".random-class".
#> Error:
#> ! Test failed

Using selenider with shinytest2

Since shinytest2 uses chromote as a backend, it can be used with selenider. selenider can be used to add more robust UI testing to shinytest2, replacing unreliable uses of AppDriver$expect_screenshot().

shinytest2 does have a few UI expectations (AppDriver$expect_text(), AppDriver$expect_html() and AppDriver$expect_js()), but these do not include the same laziness and implicit waiting that selenider provides, making them a bit less reliable.

library(shiny)
library(shinytest2)

Let’s create a simple shiny app, consisting of a shiny::actionButton() and shiny:: conditionalPanel(). The panel is shown if the button has been clicked an odd number of times, and hidden otherwise.

We would like to test that the server-side processing of the button input is done correctly, which we can do using shinytest2. However, we would also like to check that the panel is visible at the correct times, which we cannot do with shinytest2, and so we will use selenider instead.

shiny_app <- shinyApp(
  ui = fluidPage(
    actionButton("button", label = "Click me!"),
    conditionalPanel(
      condition = "(input.button % 2) == 1",
      p("Button has been clicked an odd number of times.")
    ) |>
      tagAppendAttributes(id = "condpanel")
  ),
  server = function(input, output) {
    even <- reactive((input$button %% 2) == 0)
    exportTestValues(even = {
      even()
    })
  }
)

To start a selenider session using an existing shinytest2::AppDriver object, supply it to the driver argument of selenider_session(): session <- selenider_session(driver = <AppDriver>)

test_that("App works", {
  app <- AppDriver$new(shiny_app)

  session <- selenider_session(driver = app)

  s("#condpanel") |>
    elem_expect(is_invisible)

  app$click("button")

  app$expect_values(export = "even")
  s("#condpanel") |>
    elem_expect(is_visible)

  app$click("button")

  app$expect_values(export = "even")
  s("#condpanel") |>
    elem_expect(is_invisible)
})
#> Can't compare snapshot to reference when testing interactively.
#> i Run `devtools::test()` or `testthat::test_file()` to see changes.
#> New path: /tmp/RtmpO6sHiu/st2-1ca821c9e4f4/001_.png
#> Can't compare snapshot to reference when testing interactively.
#> i Run `devtools::test()` or `testthat::test_file()` to see changes.
#> New path: /tmp/RtmpO6sHiu/st2-1ca821c9e4f4/001.json
#> Can't compare snapshot to reference when testing interactively.
#> i Run `devtools::test()` or `testthat::test_file()` to see changes.
#> New path: /tmp/RtmpO6sHiu/st2-1ca821c9e4f4/002_.png
#> Can't compare snapshot to reference when testing interactively.
#> i Run `devtools::test()` or `testthat::test_file()` to see changes.
#> New path: /tmp/RtmpO6sHiu/st2-1ca821c9e4f4/002.json
#> Test passed

Note the difference in styles: while in selenider you must specify tests explicitly, shinytest2 uses a snapshot-based approach (specifying the value that you want to test and omitting the value that you expect it to be). There are advantages and disadvantages to this approach: the tests are generally easier to create and update, but a little harder to debug.

If you want to use a snapshot-based style, you can do it manually, e.g.:

expect_snapshot(is_visible(s("#condpanel")))

However, note that the tests will no longer wait a certain period of time for the value to be correct, since the test is unaware of what the correct value is.

Using selenider with Github Actions

The complexity of using selenider with Github Actions depends on the backend that you use.

If you would like to use chromote as your backend, you shouldn’t need to make any special additions to your workflow files, and can safely use something like r-lib’s R CMD CHECK action. This is because chromote only requires chrome to be installed, which is already the case on Github’s machines.

If you want to use selenium with Github Actions, it is recommended to make use of docker. See https://github.com/SeleniumHQ/docker-selenium for more information. For example, the following lines in a Github Actions yaml file will start a selenium server (version 4.15.0), supporting Firefox, on port 4444. We recommend using the “shm-size” argument to make sure you don’t run out of memory.

services:
  selenium:
    image: selenium/standalone-firefox:4.15.0-20231108
    ports:
      - 4444:4444
    options: >-
      --shm-size="2g"

This will download Firefox and start a Selenium server on port 4444. Automating a browser with Selenium consists of two parts: the server and the client. By default, selenider_session() tries to setup both, but we can stop this from happening by using the options argument.

session <- selenider_session(
  "selenium",
  browser = "firefox",
  options = selenium_options(
    server_options = NULL, # Stop selenider from creating a server
    client_options = selenium_client_options(port = 4444L) # Use the port of the server
  )
)

The session can then by used as usual. selenider will no longer be able to close the selenium server, but this should be done automatically in the Github Action.

For more information, see how we setup our Github Actions workflow for selenium: https://github.com/ashbythorpe/selenider/blob/main/.github/workflows/R-CMD-check-selenium.yaml https://github.com/ashbythorpe/selenider/blob/main/tests/testthat/helper-session.R