Muffinman

Muffinman is not currently available for download, but this section still shows how little is required to get Muffinman to start working for your application.To Demonstrate this, I will be using the Eclipse IDE. The same task should be able to be completed in any IDE, but the image below is in Eclipse.

What we want to do is add Muffinman to the classpath of our application. Using the LoginDemo app as an example, We can open up the Run Configurations for it in Eclipse (Right Click on the java file containing your main() method and select 'Run As' -> 'Run Configurations'). Once here, We want to Add Muffinman to the classpath of the application we are testing. So, we add it under 'Bootstrap Entries' and we Move the JRE so that it is below our Muffinman dependencies. This last part is key so make sure you adjust the position of the JRE library as shown in the image below:

The above image shows quite a few seperate projects as dependencies which are added. This is since Muffinman is not packaged up and we are still working with the code in development. Normally, when the project is packaged for distribution, there would only be a single JAR file with all the needed files in it. In this case only the JAR file would show up above the JRE.

Now, When you start your application (LoginDemo in this case), Muffinman will automatically show up along with it. Thats it! you are ready to start creating scripts with Muffinman!

We're going to start by creating our first simple scripts using Muffinman. We will be using a simple demo application called 'LoginDemo' as our test application. The source code is available at the very beginning of this guide. and you can follow the previous section in this guide to get Muffinman configured with this app.

Once you have the source code, and Muffinman conifgured, start up LoginDemo and you should see both a simple login screen and the Muffinman interface like below:

If Muffinman is not appearing, then something is not configured properly, please double check with the previous sections directions.

Now, lets say we want to create a simple script to login to our application. The first thing we are going to need to do is to gather the unique ID's for each of the interface components we will need for our script. The simplest way to do this is through recording actions on these components. So, select the 'Record' tab at the top of the Muffinman interface and specify where you want your recorded script to be saved. All you need to do is click the '…' button and move to the directory where your script should be saved. Then type in the file name with a basic extension (.txt). Now, you are ready to start recording. So, click the record button. For now, All we want to do is get the component ID's which is going to consist of clicking on each component of interest: the user name field, the password field, and the login button. Once you have clicked each of these components once, click the stop button to turn off recording. Now, go open up your script and it should look something like this:

click("guis.LoginDemo..javax.swing.JRootPane..javax.swing.JLayeredPane..javax.swing.JPanel
..javax.swing.JPanel..javax.swing.JTextField0");
click("guis.LoginDemo..javax.swing.JRootPane..javax.swing.JLayeredPane..javax.swing.JPanel
..javax.swing.JPanel..javax.swing.JPasswordField0");
click("guis.LoginDemo..javax.swing.JRootPane..javax.swing.JLayeredPane..javax.swing.JPanel
..javax.swing.JPanel..Login");

All we have are the outputted click commands in the Muffinman scripting language. You can see the ID parameters being passed to specify which component is being targeted. Don't worry about what exactly each part of it means, it won't affect things at all. What is important however, is to remember which ID corresponds to which component. So, lets copy them into variables which are more easily identified. Instead of the above, replace your script with this:

userName = "guis.LoginDemo..javax.swing.JRootPane..javax.swing.JLayeredPane.. javax.swing.JPanel..javax.swing.JPanel..javax.swing.JTextField0";
password = "guis.LoginDemo..javax.swing.JRootPane..javax.swing.JLayeredPane.. javax.swing.JPanel..javax.swing.JPanel..javax.swing.JPasswordField0";
loginButton = "guis.LoginDemo..javax.swing.JRootPane..javax.swing.JLayeredPane.. javax.swing.JPanel..javax.swing.JPanel..Login";

Now, we have the same components but we are storing them in a way which is easier to use. Now that we have these components, lets create our login script. Our script is only going to consist of 3 actions: type our user name, type our password, click the login button. So, lets write the script. In the same file as your variable declarations, but below them, add the following:

type(userName,"Pete");
type(password,"petepass");
click(loginButton);

Each of these actions is pretty self explanitory, you can see we are using the variables we defined earlier and in the type actions, we are passing the values we want to type into the fields we are targeting. So, lets go back to the Muffinman Interface, click on the 'Run' tab and select the script you just created (make sure it is saved and up to date). Now, with the LoginDemo interface also open, go ahead and click 'Play All'. If everything goes as it should, the script should run and you should now be logged into the application and you should see a form Submission page.

Congratulations! You've created your first basic script using Muffinman. You've played with Recording, the scripting language, and running of scripts. In the next section, we'll take a closer look at scripts and their design, running larger groups of scripts, and building your scripts to produce useful results which can be easily obtained.

In this section we will take a closer look at different ways to set up your scripts and also Muffinman's built in report generator.

To start, I recommend taking a look at the supplied batch of test scripts linked to at the start of this page. What the archive contains is a small suite of tests for the demo application we've been working with. With this downloaded and unzipped, open up the script titled 'MAINTESTSUITE.txt'. This script contains a list of 'importAndRun' actions which are being used to import each seperate script in our test suite. Don't worry about the details of that for now, what is needed is for you to adjust the file paths used in the parameters to work on your machine. For example, what is in the parameter currently, and what works on my machine is:

"C:\\Documents and Settings\\Pete\\My Documents\\Demo\\Scripts\\[scriptName].txt"

This path should be changed to something that leads to where you are keeping the scripts. Note that if you are using Linux or Mac, the slashes will be different since the above example is using Windows. Make sure you change the path for each command in this main test script.

So, what exactly is this script doing? well, importAndRun is a built in action which will actually run another test script(a 'subscript') in a separate file at the location of the importAndRun action call. So, we are running several scripts from one central superscript. There are several benefits to this design as we will see shortly. Also, you can open up each of these smaller scripts to see what they are doing. There are a few things worth noting: we only assign variables once in a single script at the start of this suite. In doing this, we are able to create our variables once and use them in all of our smaller subscripts. This helps avoid repetition of variable assignments. Also, we are breaking each test up into a small area of functionality. each test case checks one small piece of our applications functionality. This sort of script design is nice because certain scripts can be easily understood and reused in other suites if it is needed.

Now, open up Muffinman with the LoginDemo application. Next, click the 'Add Script' button and find and select the MAINTESTSUITE.txt file that was mentioned earlier. Since this script imports and runs all the other scripts, we do not need to add any other files to run our full suite. In your script file list, you should only have a single script as below:

Now, select this script and press 'Play Selected'. You should see a variety of actions being taken on the LoginDemo UI, but it will all happen very quickly. So, how do we know what happened? Where are the results? Go ahead and click on the tab titled 'Results' at the top of the MuffinUI. You will see a list of scripts that have been run, their status, and a field to get a report about the script. Right now, we want to generate a report just to see what we ran. If you had run a simple single script, This report will generate a single html file. However, we ran a script that imported many smaller scripts into it, so our report will actually consist of multiple html files which are linked together. So, lets go ahead and double click on the "Get Report" cell for our test suite and you should be prompted with a save dialog box. Enter the name, including extension, for your report (like "suiteReport.html") and save it in a directory of your choice. NOTE: Since this saves off many files, you may want to create a new directory for your report files. Now navigate to this directory and find the root report file. it will be the one which does not have a "_sub" in the title. Opening it in a web browser should look something like this:

You can see that each of our subscripts has a link to its specific results report. This means that when we run a full suite of tests, we can have a report generated which links between all of the results for each test in the suite and allows us to easily find failures within these tests. Now, As it is, All of our test cases are passing, which is what we want. But, just to show a bit more of Muffinman's reporting, lets make one of them fail.

Open up one of the scripts, like validLogin.txt, and lets change one of the lines so we know it should fail. for instance, in my example I will change the last line to:

isDisplayed(logout2);

So, I am trying to use the 'logout2' variable, which doesn't actually exist. So, this should result in an error. Now save the script, and re-add it to your script list (remove, and add the script suite as we did before) to refresh your newest changes. Now, run the script as we did before, but this time, it should show a fail status overall. Follow the same steps as before to generate a report and opening it should show this:

Now, if you click the link for the script that is marked as failed, you will see the action which is causing the problems. it also has a message with it that reads:

"Component 'logout2' has not been registered with Muffinman, and could not be found."

So what this is saying, is that since logout2 does not actually represent a valid component, Muffinman was unable to find it. So, as you can see, when your scripts fail, Muffinman will show you what actions caused the problem and will try to give some insight as to what that problem was.

We've now had a closer look at Muffinman's report generator, and some benefits of organizing your test scripts well. The following section of this manual gives a full reference for all the supported actions and features of Muffinman's scripting language. Beyond that, you are free to go and get to testing!

The Muffinman scripting language has a number of built in actions to allow you to access components within the interface of the application being tested. When your application starts up, and while it runs with Muffinman enabled, interface components are registered with Muffinman and are given a unique ID. This ID is used within the actions below to identify which component is being targeted. As described above, the ID for a component is most easily obtained by recording an action (for instance, a click) on the component. This will cause the ID for it to be saved off into a text file as part of the script command for that action. You can then copy this value into a variable for any of your scripts and can then easily use it to access that component.

Below is a list of supported actions and language features for the Muffinman scripting language, thier syntax, and a brief description:

Click: click([ComponentID]);

The click action performs a normal left mouse button single click (both press and release) on the targeted component. If the component is not registered with Muffinman, is not displayed, or is not enabled, the action will fail.

Type: type([ComponentID], [value]);

The type action sends text to a valid text field(or text area) component. The [value] specified can be either a string literal or a variable. If the component is not registered with Muffinman, is not displayed, or is not enabled, the action will fail.

Sleep: sleep([value(in milliseconds)]);

The sleep action is an action which can be used to pause the current script for a specified amount of time. This can be useful if you need to slow your scripts down to see what is happening more clearly or if you need to pause a script while your application is going through a period of slower processing. The [value] parameter can be either an numeric integer or a variable containing such a value.

Is Enabled: isEnabled([ComponentID]);

The isEnabled action behaves pretty much exactly as you would expect. It simply checks if the targeted component is enabled or not. If the component is not enabled, or if it is not registered with Muffinman or it is not displayed, the action fails.

Is Disabled: isDisabled([ComponentID]);

The counterpart to the isEnabled action. This action is very similar but checks just the opposite. In other words, if the component is disabled, it passes otherwise it fais.

Is Displayed: isDisplayed([ComponentID]);

This actions checks if a component is visible on the screen. if the component is not visible to Muffinman, it will fail.

Drag: drag([ComponentID], startX, startY, endX, endY);

This action can be used to drag a component from coordinates (startX, startY) to coordinates (endX, endY). It simulates a mouse pressed, mouse dragged, then mouse release.

Import and Run: importAndRun([filePathToScript]);

The importAndRun action is different from whats been shown so far. What this action actually does is launch another script which is specified in the file path parameter of the action. The filepath parameter needs to be the full filepath to the script for the action to work correctly. The current script does not move on to the next action until the imported script has finished its execution. This may seem like a strange action at first, but there are a variety of reasons it is useful.

For example, this action allows easy reuse of scripts in more than one area. If there is a particular area of functionality which is used repeatedly (i.e. logging in), then you can put this in its own script and have any other script simply import it to run those actions when it is needed instead of repeating the code in many areas. Additionally, since the format of reports for scripts is HTML, we are able to easily link between the reports of related scripts. What does this mean? It means that when you importAndRun a script, the report for that subscript is automatically linked to in the details of the importAndRun action within the report of its superscript. Furthermore, this means that if you were to run a long set of scripts by simply importing them all as subscripts of one super script, you could then generate a full series of HTML reports which are already linked together with line by line report information for each script in just a couple mouse clicks. And since this is all in a standard HTML format, it can easily be shared with others or even directly uploaded to the internet to display. (examples of these reports and using this action can be seen in the 'Taking it a step further' section of this document.')

Variable Assignments: [IDENTIFIER] = [value];

There are no type declarations (i.e. int, string, etc) when using variables. Simply assign the desired value to the variable name of your choice. Currently, only integer numbers and string literals, surrounded in double quotes, are valid values.

A variable name must start with a lower or uppercase letter and can contain letters, digits, and underscores.

Arithmetic assignments: [var] = [operand1] [operator] [operand2];

The scripting Language allows basic arithmetic statements for variable assignments where [operator] can be any of +, -, /, or *. The operands can be constants or variables.

For Loop: for([val1] to [val2]; [incr]) {...}

This is a standard for loop. it loops from [val1] to [val2] by an increment of [incr] each time. The actions within the loop block (between the brackets) will be executed each time the loop iterates. the start and end values can be variables as well as constatnts so that they can be used within actions inside the loop.