QA (eng)

Test Framework build 1.0

So, my latest and greatest framework build is ready (available at BickBucket, I could share the access, but it will be no core test methods, cause they came from my work and I can’t share them). This is a link to the overall description of the very first version of it. Framework available for every QA in my company and instructions, manuals, key method descriptions and examples provided, for example:

  • Which method validates that alerts and warnings exists (with expected message) and how to close them.
  • Which one method hides/shows grid columns
  • Which one method checks special characters/too long value/empty value input into the text fields
  • Which method and how add/validates date or number fields input
  • Which one method opens row menu
  • Which one validates that value exists in the grid
  • And tens of others

I successfully made TeamCity project to run test suites for some of our components and every QA could create test and upload them to the test library:

12.png Continue reading

Standard
QA (eng)

Test Suite Report Builder Comparison

Here is a short comparison of for most popular ways to build test report for TestNG + Java framework (with comparison table in the end of the post).

Log4J

This is a pretty simple logger which throws test suite execution to the Windows Console, Eclipse Console and text file.

There is no screenshot capturing or easy navigation between multiple suites, so, this is just a huge text file.

1 Continue reading

Standard
QA (eng)

Test Framework (Java, TestNG, ANT, etc)

Introduction to Automation testing:

Testing is an essential part of a software development process. While testing intermediate versions of products/projects being developed, testing team needs to execute a number of test cases. In addition, prior to release every new version, it is mandatory that the version passed through a set of “regression” and “smoke” tests. Most of all such tests are standard for every new version of product/project, and therefore can be automated in order to save human resources and time for executing them.

Benefits of using automated testing are the following:

  • Reduction of tests’ time execution and human resources required
  • Complete control over the tests’ results (“actual results” vs “expected results”)
  • Possibility to quickly change test’s preconditions and input data, and re-run the tests dynamically with multiple sets of data

Automation workflow for the application can be presented as follows:

  • First of all it is required to identify tasks that an application has to accomplish.
  • Second, a set of necessary input data has to be created.
  • Third, expected results have to be defined in order one can judge that an application (a requested feature) works correspondingly.
  • Fourth, Executes a test.
  • Finally, Compares expected results with actual results, and decides whether the test has been passed successfully.

Goal:

The goal of this framework is to create a flexible and extendable automated testing framework, which should expand test coverage for as many solutions as possible. Framework must have input and output channels and library of methods to work with UI.

Environment Specifications:

  • Selenium Webdriver. Selenium is a suite of tools for cross-platform automated testing of web applications. It runs on many browsers and operating systems and can be controlled by many programming languages and testing frameworks. Selenium WebDriver is a functional automation tool to automate the applications. It makes direct calls to the browser using each native support for automation.
  • Eclipse IDE. Eclipse is an integrated development environment (IDE) used in computer programming, and is the most widely used Java IDE. It contains a base workspace and an extensible plug-in system for customizing the environment. Eclipse is written mostly in Java and its primary use is for developing Java applications.
  • Java.
  • TestNG. Is a testing framework inspired from JUnit and NUnit. It has extended new functionalities, which made it more powerful and easier than the other testing frameworks. It supports ReportNG (simple HTML reporting plug-in) and XLST (Graphical / Pictorial reports) plug-ins to customize or extend the default TestNG reporting style. TestNG also provides ability to implement ‘IReporter’ an interface which can be implemented to generate a Customized TestNG report by users. It has ‘generateReport()’ method which will be invoked after all the suite has completed its execution and gives the report into the specified output directory.
  • Apache Ant is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other. The main known usage of Ant is the build of Java applications.
  • AutoIT Tool used to handle Windows popups for Document Uploads and Downloads.
  • Apache POI to perform operations with excel like read, write and update the excel sheet
  • Webdriver is a driver that contains programming interface for controlling all kinds of possible actions in browser.
  • Selenium TakesScreenshot to take screenshot in case of error.
  • Log4j is a reliable, fast and flexible logging framework (APIs) written in Java, which is distributed under the Apache Software License.
  • JDBC. Java Database Connectivity (JDBC) is an application programming interface (API) for the programming language Java, which defines how a client may access a database. It is part of the Java Standard Edition platform, from Oracle Corporation.

Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Spark Dataset API implementation

Dataframes

Spark introduced Dataframes in Spark 1.3 release. Dataframe overcomes the key challenges that RDDs had.

A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a R/Python Dataframe. Along with Dataframe, Spark also introduced catalyst optimizer, which leverages advanced programming features to build an extensible query optimizer.

Dataframe Features

  • Distributed collection of Row Object: A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database, but with richer optimizations under the hood.
  • Data Processing: Processing structured and unstructured data formats (Avro, CSV, elastic search, and Cassandra) and storage systems (HDFS, HIVE tables, MySQL, etc). It can read and write from all these various datasources.
  • Optimization using catalyst optimizer: It powers both SQL queries and the DataFrame API. Dataframe use catalyst tree transformation framework in four phases,
  • 1.Analyzing a logical plan to resolve references 2.Logical plan optimization 3.Physical planning 4.Code generation to compile parts of the query to Java bytecode.
  • Hive Compatibility: Using Spark SQL, you can run unmodified Hive queries on your existing Hive warehouses. It reuses Hive frontend and MetaStore and gives you full compatibility with existing Hive data, queries, and UDFs.
  • Tungsten: Tungsten provides a physical execution backend whichexplicitly manages memory and dynamically generates bytecode for expression evaluation.
  • Programming Languages supported:
  • Dataframe API is available in Java, Scala, Python, and R.

Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Modification of the SQL ER model to NoSQL Cassandra

Original MySQL ER-model:

mysql

There is no difficulties to implement account management and security case based on UI architecture – there is no need to join tables.

First problem appeared on attempt  to implement friendship data or messages – we need to join users_link and users table for friendship data and we need to join messages and friendship data to filter messages to display here.

There are some shortcuts which will allow us to solve these problems. Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Spark and Data Formats. Introduction

This is a pretty short compilation about data formats and I will ignore JSON, XML and a lot of other formats here.

From my point of view – this one presentation is very important to understand difference between different formats (I used a lot of other data sources, just in case):

http://www.slideshare.net/StampedeCon/choosing-an-hdfs-data-storage-format-avro-vs-parquet-and-more-stampedecon-2015

CSV

The CSV (“Comma Separated Values”) file format is often used to exchange data between differently similar applications. The CSV Format:

  • Each record is one line – Line separator may be LF (0x0A) or CRLF (0x0D0A), a line separator may also be embedded in the data (making a record more than one line but still acceptable).
  • Fields are separated with commas.
  • Leading and trailing whitespace is ignored – Unless the field is delimited with double-quotes in that case the whitespace is preserved.
  • Embedded commas – Field must be delimited with double-quotes.
  • Embedded double-quotes – Embedded double-quote characters must be doubled, and the field must be delimited with double-quotes.
  • Embedded line-breaks – Fields must be surrounded by double-quotes.
  • Always Delimiting – Fields may always be delimited with double quotes, the delimiters will be parsed and discarded by the reading applications.

Example: Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Spark Introduction. RDD

Once again, the key thing about queries in Cassandra (my previous article about it) –  there is no way to do joins there. So you should be very accurate with the model of the database. Anyway, if  there is a strong need to perform relational queries over data stored in Cassandra clusters – use Spark.

Let’s start with short introduction about Spark, it is:

  • A lightning-fast cluster computing technology, designed for fast computation,
  • A unified relational query language for traversing over Spark Resilient Distributed Datasets (RDDs),
  • Support of a variation of the query language used in relational databases,
  • Not about their own database and query language – Spark is about query language and other databases (in our case – Cassandra). You can execute Spark queries in Java applications that traverse over Cassandra column families.

One pretty nice article about Spark is here: http://spark.apache.org/docs/latest/programming-guide.html

Spark and Hadoop difference

1

What to add? Continue reading

Standard