Crash Reporters (iOS)

It's recommended to disable crash reporter behavior across all Emerge features that run your app (performance testing/snapshots/order files). Typical crash reporter behaviors can interfere with our instrumentation and cause inaccurate or error results. While every app has different behaviors and ways of integrating frameworks, always disabling crash reporters is recommended and achieves better results than keeping them enabled.

To disable a crash reporter, just avoid initializing the SDK (usually in applicationDidFinishLaunching). You should still link the SDK to your app and include it in the upload, for an accurate size analysis.

You can check environment variables to determine if the app is being launched by an Emerge feature and use that to disable SDK initialization. Here's an example for Bugsnag:

let env = ProcessInfo.processInfo.environment
if env["EMERGE_IS_PERFORMANCE_TESTING"] != "1" && env["EMERGE_IS_RUNNING_FOR_SNAPSHOTS"] != "1" {
  Bugsnag.start()
}

There are a few specific reasons this is recommend:

Realistic Performance Tests

Crash reporters can have different startup behavior depending on the state they think the app is in. For example, if the app has been crashing repeatedly a crash reporter may try to synchronously log this data during app start, to have this logged before getting to the code that causes a crash. They can also behave differently when an app is first upgraded. Neither of these are the expected performance a user sees during typical use of the app, and is not what you expect to see in performance tests. Performance tests have been seen to trigger both these behaviors. Crash reporters can confuse the app being force quit programmatically at the end of your test span with a repeated crash loop. The build number embedded in the app binary can also cause them to think switching between a base/head build during a test is the app being upgraded. For both these reasons, crash reporters might take a different path during app start than typical uses get, making your perf test less reflective of actual user experience.

App Hangs

Crash reporters typically include a feature to detect hangs of the main thread. Some Emerge features intentionally use the main thread for extended periods of time. If a crash reporter detects this as a "hang" it can slow down your CI and even lead to deadlock . While causing deadlock is a bug in the crash reporter, we have seen it happen and they are notoriously hard to debug. Beyond the bug though, the extra time taken while crash reporters symbolicate threads in your app and report the hang will only slow down your CI.

Conflicting Exception Handlers

Crash reporters install exception handlers to be notified of errors in your process. Emerge also uses this API for some features, but having multiple exception handlers installed is not supported by the OS and some events end up not delivered. To prevent conflicts, crash reporters should be disabled so exception handlers installed by Emerge function as expected.