iTest 3.4 Features

[AmishP]

We have many users today that use iTest’s “run” steps to create master test cases that run other test cases.  This can be a powerful way to construct a hierarchy of tests, along with procedural logic and perhaps even some reporting functions – all to create a kind of poor-man’s test management system inside iTest.  This approach, however, requires that every test that you want to run must be explicitly called out in an EXEC.run step in a master test case.

Wouldn’t it be great if you could ask iTest to “Run all of the tests in this folder for feature X that are marked with priority > 4”?  Good news.  In iTest 3.4, we have a new kind of document called a “test suite”.  This document allows you to define one or more test groups.  Each group identifies a root folder somewhere along with a set of filters that let you decide which tests in that folder (including nested folders) that you want to include or exclude.  There are many kinds of filters letting you include or exclude tests.  And using parameters, you can define your own filters.  In this way, you can incrementally add new tests into these folders and/or adjust the parameters on certain tests and the next time that you run the suite, it will adapt to these so that the suite will automatically grow or shrink accordingly.

Of course, the next natural question is how you run these test suites.  Sure, you can kick one off from within iTest if you want.  But wouldn’t it be great if you could ask iTest to run it at a certain time – and perhaps on a recurring schedule?  This is where iTest’s new jobs and scheduling comes in.  To run a test suite, you create a new job definition file that describes the test or test suite to run and the testbed and parameters to use.  Then you ask the job scheduling system to run your job – now or on a scheduled basis.  iTest will then take care of running it for you.

You can run jobs inside of iTest.  But if you can also use the new iTestRT command-line tool to have your scheduler running all the time – without having to have iTest itself open.

So if you are constructing full regression systems, you will be able to take advantage of test suites and job scheduling to simplify the integration into a complete solution.

---

Jobs and Scheduling

Want to learn more?  Click the play button on the video to see this feature in action.

Having trouble watching this video remotely?  Download the video file

---

Test Suites

Want to learn more?  Click the play button on the video to see this feature in action.

 Having trouble watching this video remotely?  Download the video file

Message Edited by BethanyW on 06-03-2009 11:29 AM

[dclaar]

To run a test suite, you need to create a job. What does that look like? Are sessions opened in the GUI when the job is running, can you stop the job, etc? The demos completed their jobs so fast that it was hard to tell.

[KumarS]

Jobs are created by creating job description files (*.ffjd). You can do that by doing File->New or selecting a test suite file or a testcase file in the iTest explorer and right clicking and from the context menu selecting New Job. After that things should be self explanatory. Jobs can be run in the UI or using itestrt. In the UI, by default there will be no UI for sessions. But you can turn this on the jobs preference page. itestrt is completely headless - so no UI there at all.

[jus4kikz]

Hell, I think with jobs, scheduling, etc, comes something that would automatically send an email report out with <passes, failures, etc>

...

Which transitions into my task... I currently have to put together a suite of tests that are runnable via itestcli.sh's CLI.  Now the suite, I Have currently done via a bunch of either "runs" or "itestcli.sh" calls (more on that below), and I would like to know if the new functionality of iTest3.4's auto report creation and job scheduling etc could aid me in the passing back to the itestcli caller the locations of the reports created with in it.

Example  test_1.fftc, test_2.fftc, test_3.fftc

Lets say a job calls 1, 2, and 3 above, and I want to some how automatically either email them the reports, OR even just a location where the reports are placed (in a well known location)

any suggestions?

(note that I used command itestcli.sh <blah> currently because the only way I know to auto gen a report of a known location was to pass it in via the -R option.. but itestcli.sh when ran with in itest itself is uglier and unlinked... than when simply calling "run" ...)

so long story short, 

How would one recommend an automated way to send link back to the reports autogenerated by the new 3.4 feature?

(dont know if there is an "info-report" for the specific report that will be created by a specific test that it can return to its caller?)

thanks

 -Anthony

[jus4kikz]

hmmm, I see actually that the reports that are autogenerated, their names are created with the timestamp of the begining of execution.... maybe a hack but...

does any one know of a way I can query to get that timestamp so then I can build the string for the predicted report (if it works as expected?)

thanks

 -Anthony

[PaulD]

There is a preference that controls the naming convention of the reports that are generated.  By default, these include the timestamp to make them unique.  But I believe you can reconfigure the format string for these however you want.  As long as they are going into an empty directory, then you won't need the timestamp, and the names will then be predictable.

[dclaar]

I don't know about the new stuff, but the old stuff has //testcase/testReport when you run itestcli with --xmlFormat option. That's what I use to grab it.

[jus4kikz]

hmm, yea, I could do this if I have the automation create a new directory and then just put them in there... is there a way to tell the automatically saved reports to go some where dynamically?  I see that this new feature can be set via Preferences..

and, also, do you know of a query that can return the time started for these .fftc's?  The military time stamp is appended to the name, and I could just use those and I'd be fine with sending the user an email ... if I had some way of just propegating a bunch of them backwards from an itestCLI call to a bunch of runs...

[jus4kikz]

wow, sorry for this thread guys.... because I have to call initially from the itescli....

I found a way to have it put all reports recursively in any directory I want, (using the -r and the -sr params)

I can just have it all done from there!

 -r,--testreport <dir_or_file>              Directory name to store test
                                            report in or filename of the test report
 -rw,--refreshWorkspace                     Refresh workspace before
                                            executing
 -sl,--suppressLiveIssues                   Suppress printing of execution
                                            issues as they are generated
 -sr,--subreports                           Generate test reports for this
                                            test case and for all children (recursively)
 -ss,--showSplash                           Display the iTest splash
 

sorry for the dumb questions, and thanks for the info guys

Message Edited by jus4kikz on 05-29-2009 08:47 PM