Need some HowTo examples

[QAutomatron]

First of all, thank you for this project. I want to try it in our app and need some "howto" examples.
I already add Demo to our UI target and it's work. (or it should be in Unit Test?)

Now i have a couple of questions:

1. How to set permissions
2. How to set Geolocation
3. How to set timeouts for waits\polling
4. How to cleanup app (keychain and other data)
5. How to assert different elements states (visibile, not visible, exist, not exist) and the same action but with wait (waitForVisibility)
6. Element selection, is it only id, label, value and visibleText based ?
7. What the difference between: element in element.id == "label" and $0.id == "button" ?

I will be grateful for the help.

[artyom-razinov]

Hi, you are the first person outside of our company who checked out this project (wow). This project is designed to be open source and highly customizable without forking, however, it really lacks docs, demos and examples. The demo now just shows how to link Mixbox to your project with very few examples of usage of the framework.

1. How to set permissions

You can add permissions: ApplicationPermissionsSetter to your base XCTestCase class (in our examples we name it TestCase.

(The code from Tests project, which is now a better demo than Demo:

Mixbox/Tests/BlackBoxUiTests/Support/TestCaseUtils.swift

Line 91 in a088341

permissions = applicationPermissionsSetterFactory.applicationPermissionsSetter(

)

permissions = applicationPermissionsSetterFactory.applicationPermissionsSetter(
    bundleId: XCUIApplication().bundleID,
    displayName: "Your app name as in home screen aka springboard",
    testFailureRecorder: testFailureRecorder
)

Factory:

let applicationPermissionsSetterFactory = ApplicationPermissionsSetterFactory(
    notificationsApplicationPermissionSetterFactory: FakeSettingsAppNotificationsApplicationPermissionSetterFactory(
        fakeSettingsAppBundleId: "ru.avito.NotificationPermissionsManager"
    ),
    tccDbApplicationPermissionSetterFactory: TccDbApplicationPermissionSetterFactoryImpl()
)

What is Fake Settings App and how to setup it: https://github.com/avito-tech/Mixbox/tree/master/Frameworks/FakeSettingsAppMain

If you do not want to set up notification permissions (but want to setup every other permission), you can pass AlwaysFailingNotificationsApplicationPermissionSetterFactory to notificationsApplicationPermissionSetterFactory.

2. How to set Geolocation

Use LocationSimulator (and LocationSimulatorImpl), as every other util we tend to put it in TestCase base class in our tests in our company. And to instantiate every class with its dependencies we use a class called TestCaseUtils (should be made by yourself, as in Tests project), because it a handy option to create dependencies for base XCTestClass, which has 2 init (constructors), which is bad, but there is an option to initialize dependencies with only one line: private let testCaseUtils = TestCaseUtils() inside TestCase. And then expose everything you need to public interface of your TestCase:

class TestCase: XCTestCase {
    private let testCaseUtils = TestCaseUtils()
    
    var locationSimulator: LocationSimulator {
        return testCaseUtils.locationSimulator
    }
}

Then in your tests it will be easy (assuming you have your own Locations class with constants for different locations, it is purely your choice to use a class or magic numbers):

locationSimulator.simulate(location: Locations.moscowCityCentre)

3. How to set timeouts for waits\polling

Every check and action contains default timeout (which is not configurable globally yet and equals 15 seconds).

You can override timeout with withTimeout modifier of page object element:

pageObjects.search.searchButton.withTimeout(30).assert.isDisplayed()
pageObjects.search.searchButton.withoutTimeout().assert.isDisplayed() or pageObjects.search.searchButton.withTimeout(0).assert.isDisplayed()

4. How to cleanup app (keychain and other data)

Unfortunately, we didn't open-source this feature yet. For example, clearing keychain is tricky and we use our proxy in our code, which was hard to open-source (in a hurry at that moment), but not impossible. I'll do it soon (because you asked about that).

5. How to assert different elements states (visibile, not visible, exist, not exist) and the same action but with wait (waitForVisibility)

If I understood you correctly, you asked "how to wait for visibility before each check" - it is done by default, but can be turned off.
If it is about which checks are implemented, see this

All actions:

func tap(...)
func press(...)
func setText(...)
func swipeToDirection(...)
func swipeUp(...)
func swipeDown(...)
func swipeLeft(...)
func swipeRight(...)

Deprecated actions:

func typeText(...)
func pasteText(...)
func cutText(...)
func clearText(...)
func replaceText(...)

All checks (syntactic sugar):

isDisplayed(...)
isNotDisplayed(...)
isInHierarchy(...)
hasValue(...)
becomesTallerAfter(...)
becomesShorterAfter(...)
isEnabled(...)
isDisabled(...)
func hasText(...)
func textMatches(...)
func containsText(...)
func checkText(...)
func hasAccessibilityLabel(...)
func accessibilityLabelContains(...)
func checkAccessibilityLabel(...)

Basic:

func isNotDisplayed(...)
func matches(...)

Getters (will be replaced soon with more generic syntax):

func visibleText(...)

---

Example of generic check (very complex example):

pageObject.map.pin.matches { element in
    let hasProperCoordinates = element.customValues["coordinates", Coordinates.self].isClose(to: Locations.moscow)
        || element.customValues["coordinates", Coordinates.self].isClose(to: Locations.moscow)
}

In this check you can pass matcher. Matchers are highly costomizable, it this example the matcher is built by a builder, but you can use classes instead:

pageObject.map.pin.matches { _ in
    HasPropertyMatcher(
        property: { $0.visibleText },
        name: "visibleText",
        matcher: EqualsMatcher("Expected text")
    )
}

I described matchers in the next answer:

6. Element selection, is it only id, label, value and visibleText based ?

There are currently those fields in every element that can be used in matcher:

public protocol ElementSnapshot: class, CustomDebugStringConvertible {
    // Common (can be retrieved via Apple's Accessibility feature):
    
    var frameOnScreen: CGRect { get }
    var elementType: ElementType? { get }
    var hasKeyboardFocus: Bool { get }
    var isEnabled: Bool { get }
    
    var accessibilityIdentifier: String { get }
    var accessibilityLabel: String { get }
    var accessibilityValue: Any? { get }
    var accessibilityPlaceholderValue: String? { get }
    
    var parent: ElementSnapshot? { get }
    var children: [ElementSnapshot] { get }
    
    var uikitClass: String? { get }
    var customClass: String? { get }
    
    // Mixbox specific (for apps with Mixbox support):
    
    var uniqueIdentifier: OptionalAvailability<String> { get }
    var isDefinitelyHidden: OptionalAvailability<Bool> { get }
    var visibleText: OptionalAvailability<String?> { get }
    var customValues: OptionalAvailability<[String: String]> { get }
}

Matchers don't simply check fields, they are also provide reports, so we use Matcher abstraction. Matchers can be constructed with primitives like EqualsMatcher and combined with matchers like AndMatcher or OrMatcher.

But in most of the cases you should use builders, they are much more convenient and the code is much more readable with them.

Example of using builder when writing a locator for page object element (very complex example):

    public var navBarBackButton: ViewElement {
        return any.element("кнопка назад в navigation bar")  { element in
            if ProcessInfo.processInfo.operatingSystemVersion.majorVersion >= 11 {
                return element.id == "BackButton" || element.label == "Back"
            } else { // iOS 9, 10
                let matcherById = element.id == "BackButton"
                
                let isLabel = element.isInstanceOf(UILabel.self)
                let isStaticText = element.type == .staticText
                let isButton = element.type == .button && element.isSubviewOf { element in
                    element.id == "NavigationBar" // I am not sure it is needed, but I'm afraid to match extra element without a check that it is a navigation bar
                }
                let isOfProperType = isLabel || isStaticText || isButton
                
                let matcherByText = element.label == "Back" && isOfProperType
                
                return matcherById || matcherByText
            }
        }
    }

All properties and functions of matcher builder is listed here:

Mixbox/Frameworks/UiTestsFoundation/PageObjectElement/Locator/Builder/ElementMatcherBuilder.swift

Line 25 in a6ceb38

public final class ElementMatcherBuilder {

Note that you can always write extensions to the builder and/or use your custom Matcher subclasses.

7. What the difference between: element in element.id == "label" and $0.id == "button" ?

It's swifts syntax. No difference. This: { arg in use(arg) } is equal to { use($0) }

---

Hope this helps. I'll make more docs and a better demo, but don't want to name any deadlines on that.

[artyom-razinov]

(or it should be in Unit Test?)

No

[QAutomatron]

Thank you for answers. I already tried other iOS frameworks (like KIF and EarlGrey) and now want to try this one, but due to lack of time and documentation it's not so easy as i thought.

1. How is permission set should work?
   Let assume i use Demo as a basic, so i add your example code to TestCaseDependencies

Now it looks like this:

final class TestCaseDependencies {
    let application = SBTUITunneledApplication()
    let pageObjects: PageObjects
    
    let permissions: ApplicationPermissionsSetter
    let applicationPermissionsSetterFactory: ApplicationPermissionsSetterFactory
    let testFailureRecorder: TestFailureRecorder
    
    init() {
        let currentTestCaseProvider = AutomaticCurrentTestCaseProvider()
        let screenshotTaker = XcuiScreenshotTaker()
        let stepLogger = XcuiActivityStepLogger(originalStepLogger: StepLoggerImpl())
        let snapshotsComparisonUtility = SnapshotsComparisonUtilityImpl(
            // TODO
            basePath: "/tmp/app/UITests/Screenshots"
        )
        let snapshotCaches = SnapshotCachesImpl.create(cachingEnabled: false)
        let applicationProvider = ApplicationProviderImpl { XCUIApplication() }
        
        let testFailureRecorder = XcTestFailureRecorder(
            currentTestCaseProvider: AutomaticCurrentTestCaseProvider()
        )
        self.testFailureRecorder = testFailureRecorder
        
        let applicationPermissionsSetterFactory = ApplicationPermissionsSetterFactory(
            // TODO: Tests & demo:
            notificationsApplicationPermissionSetterFactory: AlwaysFailingNotificationsApplicationPermissionSetterFactory(
                testFailureRecorder: testFailureRecorder
            ),
            tccDbApplicationPermissionSetterFactory: TccDbApplicationPermissionSetterFactoryImpl()
        )
        self.applicationPermissionsSetterFactory = applicationPermissionsSetterFactory
        
        permissions = applicationPermissionsSetterFactory.applicationPermissionsSetter(
            bundleId: XCUIApplication().bundleID,
            displayName: "AppNamePrelive",
            testFailureRecorder: testFailureRecorder
        )
        
        pageObjects = PageObjects(
            pageObjectDependenciesFactory: XcuiPageObjectDependenciesFactory(
                interactionExecutionLogger: InteractionExecutionLoggerImpl(
                    stepLogger: stepLogger,
                    screenshotTaker: screenshotTaker,
                    imageHashCalculator: DHashV0ImageHashCalculator()
                ),
                testFailureRecorder: XcTestFailureRecorder(
                    currentTestCaseProvider: currentTestCaseProvider
                ),
                ipcClient: SbtuiIpcClient(
                    application: application
                ),
                snapshotsComparisonUtility: snapshotsComparisonUtility,
                stepLogger: stepLogger,
                pollingConfiguration: .reduceWorkload,
                snapshotCaches: snapshotCaches,
                elementFinder: XcuiElementFinder(
                    stepLogger: stepLogger,
                    snapshotCaches: snapshotCaches,
                    applicationProviderThatDropsCaches: applicationProvider
                ),
                applicationProvider: applicationProvider,
                applicationCoordinatesProvider: ApplicationCoordinatesProviderImpl(applicationProvider: applicationProvider),
                eventGenerator: EventGeneratorImpl(applicationProvider: applicationProvider)
            )
        )
    }

Now from test i can call: testCaseDependencies.permissions.geolocation.set(.allowed)
But nothing happens.

So test itself (i moved launch() to setUp):

class DemoTests: TestCase {
    func test() {
        testCaseDependencies.permissions.geolocation.set(.allowed)
        
        // Test steps here
    }
}

4. We currently have in app method to do so, but may be there is better approach

As for 2,3,5,6,7 - Nice, thanks, will try to use this.

• Another question is how to setText or typeText into hiddentext field? I can do it using XCUITest, but Mixbox want element to be completely visible before typing

[artyom-razinov]

But nothing happens.

What should happen? I do not see code in your test that would check that geolocation permissions were set. It works silently.

Also there might be problems on Mojave with permissions, our testing farm uses High Sierra.

We currently have in app method to do so, but may be there is better approach

Our approach is fast (milliseconds fast), however we don't have blackbox alternative (using UI of iOS) as a fallback.

Another question is how to setText or typeText into hiddentext field? I can do it using XCUITest, but Mixbox want element to be completely visible before typing

Mixbox can not interact with anything hidden (as a real person). Autoscrolling is enabled by default for every action or check (can be manually disabled). It is not tested with UITableView (we use UICollectionView). If scrolling is not working, there is a workaround - accessibleViaBlindScroll modifier:

    public func categoryCellByTitle(title: String) -> ViewElement {
        return accessibleViaBlindScroll.element("тайтл ячейки категории \(title)") { element in
            element.label == title && element.id == "TTTAttributedLabel"
        }
    }

It is a workaround, it is not reliable, because it doesn't know where the element is. It tries to scroll view randomly until desired element appears.

[artyom-razinov]

But nothing happens.

What should happen? I do not see code in your test that would check that geolocation permissions were set. It works silently.

Also there might be problems on Mojave with permissions, our testing farm uses High Sierra.

We currently have in app method to do so, but may be there is better approach

Our approach is fast (milliseconds fast), however we don't have blackbox alternative (using UI of iOS) as a fallback.

Another question is how to setText or typeText into hiddentext field? I can do it using XCUITest, but Mixbox want element to be completely visible before typing

Mixbox can not interact with anything hidden (as a real person). Autoscrolling is enabled by default for every action or check (can be manually disabled). It is not tested with UITableView (we use UICollectionView). If scrolling is not working, there is a workaround - accessibleViaBlindScroll modifier:

    public func example(exampleArg: String) -> ViewElement {
        return accessibleViaBlindScroll.element("example") { element in
            element.visibleText == exampleArg && element.id == "exampleId"
        }
    }

It is a workaround, it is not reliable, because it doesn't know where the element is. It tries to scroll view randomly until desired element appears.

[QAutomatron]

Thanks, got it.

What should happen? I do not see code in your test that would check that geolocation permissions were set. It works silently.
Also there might be problems on Mojave with permissions, our testing farm uses High Sierra.

Hm, Mojave could be an issue. Currently after calling testCaseDependencies.permissions.geolocation.set(.allowed) i still can see alert to set location permission in my app. So may be there is an option to debug this, at least get some message in log or something like that.

Mixbox can not interact with anything hidden (as a real person).

This field is actually on screen. It's some custom field implementation with facade like dots and underlines (like typical pin screen). And XCUITest can sendkeys to it, but Mixbox tries to scroll and find it and fails.

Actual element tree looks like:

• OneCharacterTextField
• OneCharacterTextField
• OneCharacterTextField
• OneCharacterTextField
  • UIView
  • UIView
    • label
    • UIView
• UITextField <= This one with id and get text from x4 OneCharacterFields and accessible from XCUI

And it is not actually hidden, but also not visible as classic field due to facade

I also have a question about network stubbing. You use SBTUITestTunnel, is there any internal proxies to point app on it or some helpers to use stubs? For example if i want to mock only selected endpoints and don't wan't to point all my app into the stub-server.

[artyom-razinov]

I understood your UI. But didn't understand the point of usage of hidden UITextField from tests. It should not be accessible from a test. It is a private implementation, not a user interface.

There is an option to "type" into a field that is not text view. But it should be visible. For example, you can type in first OneCharacterTextField, then to second OneCharacterTextField. Or just into a first OneCharacterTextField (I think it allows to type 4 digits at once). Are they text views/text fields?

Also, if setText doesn't work (it works via cmd+A + cmd+V), try typeText, it enters characters one by one. It may be slow, because it waits until element gains focus, which will not happen. Or it might even fail, I don't remember. pasteText now doesn't contain this check (so actions now are a bit chaotic, but it will be improved soon, there is a branch called split-actions with customizable actions, but they are not ready).

I also have a question about network stubbing
is there any internal proxies to point app on it or some helpers to use stubs?

Facade is not open sourced. You can use raw SBTUITestTunnel interfaces. We plan to get rid of SBTUITestTunnel in a month or two and open source mocking interfaces for Black Box and Gray Box tests (Gray Box tests are in development and do not work).

Interface of the facade looks like this now:

networking.stubbing
    .stub(fullUrl: ".*/path/\(arg)/path/path")
    .thenReturnValue(value: myEntityStub(entityId: id, entityOtherArg: 42))
    
networking.stubbing
    .stub(method: MyApiMethod())
    .thenReturn(file: "my_api_response.json")

Interfaces will also be changed. SBTUITestTunnel functionality is tool limited for us. For example, it doesn't support proxying and modification of real responses.