Preliminary Report on Aland

Database Module parses these files and saves data to the database. The database is created at Database Module installation at a local or remote MS SQL server.

Presentations Module is used to create presentations. It has a few common components as Text, SmartText, Background, Image, Shapes, etc. used to arrange the presentation content. A presentation consists mainly of various diagrams; that provide for algorithms that allow for building them when addressing the database with various requests. The mode of a presentation viewing is totally controllable.

Thus, presentation that is created in Presentations Module allows for displaying economic performances in various contexts and is grounded on the by Database Module’ created database of files that are submitted by various companies in different formats.

The following testing types were applied to the Presentations Module and the Database Module:

• Functional testing (partially automated functional testing)
• Stress testing
• Usability testing
• Load testing
• Installation testing
• Documentation testing (configuration audit)

Aland project a dedicated team was allocated (one test engineer per Presentations Module and one per Database Module) that was busy with the project almost full-time. Many related aspects were tested together (e.g., Custom format, Slicing, packets, etc.). For automation works, occasionally, we engaged an engineer from the automated testing team.

As a result, after fixing the bugs revealed during these types of testing, the application was made compliant with the customer requirements and have the application behave correctly in case of incorrect user input.

Presentations Module. The first finished functionality of the Presentations Module was common components designated for presentations management and composition. Initially, common components were implemented in partial functionality, including the partially finished Properties window. After a few builds the complete functionality was reached. The product was bur raw and multiple bugs prevented its normal operation. As soon as the common components functionality was totally completed, test cases were developed to cover all the functionality of the common components (location on the screen, context menu items functionality, options window: modification, saving, cancellation, default). These were these test cases that were automated – scripts run over all the four hundred test cases in an hour and the results are saved in a log.

During the first few runs scripts revealed many bugs. Yet, after these were fixed and this functionality debugging, the developer team set to Presentations Mode development and then scripts were applied for the purpose of regression testing once in a few builds. Since this functionality does not further intersect with the implemented one, the new bugs were revealed but few during next regression testing.

When testing the Presentation Module, investigated were navigation across a presentation, shifting between the presentation play mode and editing mode, displaying and editing the available objects in the presentation, common controls operation when in presentation mode, masterslide components display.

The most time consuming stage of the development process was in diagram development. Each of ca. 50 diagrams, apart to the common (pattern) functionality, implements a specific request to a relevant database. Apart to testing common options of the diagrams (e.g., font, size, alignment, etc.), it was necessary to understand their algorithms and check for it to display the database request results correctly, which was performed successfully.

Simultaneously to checking the diagrams, additional functionalities were put into development: Data Source Manager, Aliases, Colors, Updater, and Slicing. The testing of their functionality was performed in normal mode. It is worth to mention our testing method of slicing in packages. For this purpose, we imported different data into empty databases on a few PCs. After opening the package we checked for the availability of these and those data in the open database by means of building the diagram. For the purpose of software versions auto update we created a dedicated server on one of our servers, while in a config file we changed the default server settings. Custom format testing was held simultaneously to Database Module testing; both test engineers participated (since this particular testing task includes both data correct import to database, and the imported data correct display in the relevant Presentations Module diagrams).

During the final stage of testing, in each build, we tested all the diagrams that are being build, main menu items functionality, opening presentations with saved diagrams. Once per several builds, scripts were run and analyzed.

Database Module. We started testing of this module with Scan type. In the first builds of Database Module, mapping and importing were performed slowly – the machine was busy a few hours to complete importing. It took a few builds to normalize Scan data importing.

Having completed debugging Scan data functionality, we started working in the two directions: other data types importing and intermediate import steps (plus supporting functionalities). After importing all the types (apart the Custom type), the importing was automated. The automation is grounded on the import profile functionality. Manually, there were created profiles that render initial data for different types in all possible combinations. A script loads them one-by-one, passes the all the import stages (ignoring all the undefined fields while mapping) and evaluates import results based on the information given to the Database Module log. This script was used until the release for the purpose of regression testing of all data types import.

Without any troubles there were tested: Field Map (including the multistage testing of added/deleted field usage while mapping), wizard functionality, hierarchy validation functionality (which was one of the most complicated testing jobs within Database Module, automation thereof was problematic, while Child Sum error was ejected off the point; to eliminate this we had to check the real numbers in database).

Apart to Database Module use cases running, data import was tested in the contest of data saving, i.e. the initial data was tested to be saved to the database. New user changes were also checked in the database. In the course of attentive and hypercorrect work with the database we produced many suggestions on improving the database operability. Also, enclosed user measurements were tested to be saved in the database.

It cannot but be noticed, but testing of Invalid Data Rules. We have developed the complete set of conditions that allowed for testing the filter complete functionality the filter. For this purpose we analyzed the source files content; the database was uploaded for every trial, after filtering database content was checked. The complete set of these tests was performed for each type of data. In the meantime, we discovered a faulty operation of one of the types, and discovered some bugs; besides we generated many suggestions of the functionality improvement. We created an exception coverage table for all data types in Database Module; the table cases were run for each build on the project final stage and allowed for the exception processing system’ correct functioning in the release build.

Presentations Module. The exception processing policy was developed during the coding process. Initially, evident incorrect user actions. At the same time an incorrect file could be tried to open as presentation, and a system error occurred. Bugs occurred when attempting to load non-format files to OLE containers (e.g., Excel, PPT, Video, Audio).

A few incorrect actions were found when working Tchart component options (although Tchart, being an external component, was out of the scope of our investigation, which was negotiated with the developers). Thus, Tchart was not given a special attention and debugging. Although the few bugs discovered were described.

Saving patterns and folders to work with äëÿ slides also appeared to produce some bugs when stress testing. As a result of extended stress testing we discovered a few bugs that when debugged allowed for the application to operate correctly at user created stress loads.

Database Module. When testing Database Module we also applied stress testing that allowed to reveal a few bugs. In the case the specifics of the bugs was of another kind. The majority of stress cases when testing Database Module were connected with incorrect data input for import (or incorrect input data entering). As example of the cases we can mention, when importing, defining wrong types of input data, mistakes in the different data types’ formats, etc. I.e., exception processing system was not thought thoroughly out. All the revealed bugs were documented, so after their debugging the data input subsystem became much more operable.

Stress testing was continued for intermediate import pages. We tested mapping of the fact with test in a source file. For some data types it appeared to give a faulty result. Besides, stress testing when mapping database column containing test data, being tested as measurement, also gave a faulty result in case of some source data (type sets for the two mapping exceptions are partially overlapped).

User measurements adding functionality was put under large scale stress testing. Initially, its exception processing system was not thought out thoroughly. Standard stress tests were completed for the records profile page processing.

In Presentations Module, there were implemented diagrams, while in their options’ names one of the characters was underlined. In other words, when holding Ctrl and the character simultaneously the focus moved to the control. (This was implemented in the previous version of Presentations Module.) In the new version the characters stayed underlined, yet, hot keys did not work.

In Presentations Module, dialog box size was optimized: to balance the controls size and form space. The same was done for the Database Module boxes.

We noticed, in Database Module, the created import profile cannot be deleted. It was motivated by the functionality unprovided by the specifications. After proper motivation of the necessity to fix the functionality, it was added to the Database Module.

Presentations Module. With this application, load testing was performed in a few aspects. First, we tested dynamics of operations when working with presentations containing large number of slides with various content (diagrams, blinking text, OLE components). It came out that when slides exceed a few dozens (30 to 40 slides) the presentation slows down. At this, presentation viewing and editing was handicapped; other applications were also hardly operable. There were made suggestions on optimizing memory usage, which was done.

Apart to this, OLE components of the Presentations Module have undergone load testing. We revealed small bugs. Partially, they occurred due to OLE components hold the process and Presentations Module processes documents incorrectly. Besides, Excel and PPT components, appeared to terminate processes incorrectly when loading files off. This was described and fixed timely.

Database Module. Since it was not defined as multiuser system, the Database Module was assumed a local application. Thus, loading it with simultaneous connections was not held. (For simultaneous connections load testing we installed the database on a dedicated server; three engineers worked with it simultaneously). Load testing of Database Module was concentrated on testing large files import to be correct.

Testing the Database Module and Presentations Module installation wizards revealed few errors (mainly in interface and typos). Yet, apart to installation wizards, the package includes a database installator which was rewritten a few times, and thus, needed special attention. We performed compete installation testing on a clean OS image, which revealed SQL server installation problem. Hence, malfunctions were described and fixed.

Documentation testing was one of the finishing testing techniques applied for the Aland product and logically completed the extend scale of the testing types and techniques applied for quality assurance of the software in its various functionality aspects.

The QA team worked remotely of the development team, and lacked the possibility to participate in ‘live’ meetings, thus, having obscure vision of some development decisions made. On the other side, we can’t but admit the fact, that if we were working side-by-side, we would have overlooked certain amount of mistakes. This was due to us having allocated a dedicated server for the database, and a separate test database, both created for the purpose of remote testing. As a result we discovered many bugs that simply did not occur in the developer database. Hence, we revealed and submitted for fixing many bugs that simply could not be discovered should we work in the same location. It is our remoteness and even detachment allowed us for the quality, independent testing (independent both from the software environment and pressure towards decision-making and working methods choice), and deep-reaching testing of software.

This is situation is very similar to the one when performing offshore outsourcing QA and testing jobs and can demonstrate our best performance and efficiency for such jobs.

For Aland project a dedicated team was allocated (one test engineer per Presentations Module and one per Database Module) that was busy with the project almost full-time. Many related aspects were tested together (e.g., Custom format, Slicing, packets, etc.). For automation works, occasionally, we engaged an engineer from the automated testing team.

The QA team worked remotely of the development team which allowed for the quality, independent testing, both from the software environment and pressure towards decision-making and working methods choice, and deep-reaching testing of software. This situation is very similar to the one when performing offshore outsourcing QA and testing jobs and can demonstrate our best performance and efficiency for such jobs.

The following testing types were applied to the Presentations Module and the Database Module:

• Functional testing (partially automated functional testing)
• Stress testing
• Usability testing
• Load testing
• Installation testing
• Documentation testing (configuration audit)

The testing work performed was deep reaching and quality. We researched the database completely, analyzed all the functionalities, continually held the regression testing which was made possible due to automated testing scripts.

The functional testing revealed that major functionalities work correctly with all possible specifications to the product (user requirements, GUI design requirements, and other requirements). The regression testing held in application modules was performed on qualitatively: all once revealed bugs were re-tested for occurrence.

Load testing revealed the application could respond correctly to any user actions.

Usability testing and installation testing allowed for decent tuning application GUI and the process of their installation (including the complete installation on a ‘clean’ operation system).

The documentation testing revealed a few incompliances of the developed software to customer requirements (for the both of Database Module and Presentations Module) that were timely liquidated.

The stress testing allowed for debugging the system of off-normal situations processing and make it react adequately for the most of user actions (including faulty ones).

About Bug Huntress QA Lab: The company was founded in 1998 as an independent software development and testing company. Wide experience in QA gained by the team and high demand on software testing services transformed the QA department into a new brand - Bug Huntress QA Lab (www.bughuntress.com). Bug Huntress QA Lab in cooperation with its QArea owner-company has developed a variety of cross-platform software solutions for business, security, healthcare and gaming industry. It is also known for its testing automation software for handheld devices.