Although there is much useful information on the official BrowserStack website, you will certainly have a lot of questions when you run tests for the first time. I will try to explain how to deal with it based on a simple example which uses Jest and Selenium.

There is no problem in running Jest with BrowserStack, despite the lack of documentation or examples.

NOTE! In order to run Selenium tests locally you will have to download and install approperiate driver, which is geckodriver in this case. It may be downloaded directly from official Mozilla repository .

Solution

The following solution will use Automate with Local Testing for running fully automated tests on selected version of OS and browser.
In order to simplify things a bit, I assume that you're working on a Unix system (e.g. Ubuntu, Mac OS X). Windows commands will be different only when it comes to BrowserStackLocal executable.

1. What is BrowserStackLocal?

After reading https://www.browserstack.com/automate/node you may already know that BrowserStackLocal is "something" that can be useful in tests.
Unfortunately, I didn't find any clear information about how to use it for integration with test (maybe I didn't look carefully enough).

For starters, just run downloadedBrowserStackLocal to check if it works. You will have to provide the --key argument with your BrowserStack key (as in official docs) e.g. Asdafklwe12312kldfasfmaMD:

./BrowserStackLocal --key [YOUR_BROWSERSTACK_KEY]

As a result, you will get some output. Exact messages doesn't matter, just make sure that there are no errors at this point.
Now run it with additional --folder $(pwd).

./BrowserStackLocal --key [YOUR_BROWSERSTACK_KEY] --folder $(pwd)

As you can see on the output, the contents of the current folder will be available for your tests under unique URL http://[YOUR_BROWSERSTACK_USERNAME].browserstack.com where [YOUR_BROWSERSTACK_USERNAME] is your BrowserStack username.

2. Adjust/rewrite tests

By default, our test is run using the local Selenium webdriver e.g. geckodriver (for Firefox). We would like to run it with remote BrowserStack OS and browser. Exact BrowserStack webdriver settings are defined inside capabilities object.
Here is an example:

Because local files are not directly available in the Browser Stack, you can access them only through a webserver (which serves files from BrowserStackLocal--folder). This makes sense if you think about running tests on multiple OSes because thanks to this webserver you don't have to copy files from one machine to the other, all you have to do is access the internal URL.

At this stage, if you manually connect to BrowserStack using ./BroserStackLocal --key [bs_key] --folder $(pwd) executed in the same folder as the test files, you should be able to completet all tests.

NOTE! BrowserStack Selenium tests may require setting a Jest timeouts (default is 5s). This is especially true for connecting and disconnecting which tends to be time consuming process. I would advise something around 10-15s.

If you clone example repository initiate BrowserStackLocal with test/ directory inside --folder argument and then run yarn test:manual to view the results.

3. Automate BrowserStackLocal connection

Going further, it's time to go for more automated/autonomous approach.
If you don't want (and probably you should not) to implement NodeJS wrapper for ./BrowserStackLocal, use the official one browserstack-local-nodejs.
This package will handle the download of approperiate binary files for you and can serve as an interface between your tests and the BrowserStackLocal executable.

To be honest I don't like the browserstack-local-nodejs package at all (callbacks, hard to debug, unused variables and more), but since this the official BrowserStack repository, we can hope that it will be maintained.

Let's continue with examples.

Here is how you may wrap start and stop methods from browserstack-local-nodejs to make them work with my beloved async/await syntax.

NOTE! At least in my case the disconnection after testing takes really long, which is why I set 20 seconds as a afterAll timeout.

NOTE! If the waiting time is too short, you'll probably run into this message: Jest did not exit one second after the test run has completed. When it appears, check your system process/task manager and kill BrowserStackLocal process(es) which may not close/stop correctly.

This is it. If you have any questions do not hestitate to ask and check out a sample repository that should help you a lot (I hope).