Java, NoSQL, SQL, REST API and other scary words

Most popular and simple Search and Sort algorithms

Binary Search

Given a sorted array arr[] of n elements, write a function to search a given element x in arr[].

  1. Compare x with the middle element.
  2. If x matches with middle element, we return the mid index.
  3. Else If x is greater than the mid element, then x can only lie in right half subarray after the mid element. So we recur for right half.
  4. Else (x is smaller) recur for the left half.

Search an element in a sorted and rotated array

Input  : arr[] = {5, 6, 7, 8, 9, 10, 1, 2, 3}; key = 3

Output : Found at index 8

1) Find middle point mid = (first_element + length)/2

2) If key is present at middle point, return mid. if (arr[mid] == key) return mid;

3) Else If arr[l..mid] is sorted

  1.    a) If key to be searched lies in range from arr[l]

      to arr[mid], recur for arr[l..mid].

  1.    b) Else recur for arr[mid+1..r]

4) Else (arr[mid+1..r] must be sorted)

  1.    a) If key to be searched lies in range from arr[mid+1]

      to arr[r], recur for arr[mid+1..r].

  1.    b) Else recur for arr[l..mid]

Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Array Algorithms

Reverse an array or string

Input: 123

Output: 321

1) Initialize start and end indexes (start = 0, end = n-1).

2) In a loop, swap arr[start] with arr[end] and change start and end as follows (start = start +1; end = end – 1)

Given an array A[] and a number x, check for pair in A[] with sum as x

1) Sort the array in non-decreasing order. sort(A, 0, arr_size-1);

2) Initialize two index variables to find the candidate elements in the sorted array.

   (a) Initialize first to the leftmost index: l = 0

  (b) Initialize second  the rightmost index:  r = ar_size-1

3) Loop while l < r.

   (a) If (A[l] + A[r] == sum)  then return 1

   (b) Else if( A[l] + A[r] <  sum )  then l++

   (c) Else r–

4) No candidates in whole array – return 0

/* The main function that implements QuickSort() arr[] –> Array to be sorted, low  –> Starting index, high  –> Ending index */

static void sort(int arr[], int low, int high)

{

    if (low < high)

    {

        /* pi is partitioning index, arr[pi] is now at right place */

        int pi = partition(arr, low, high);

        // Recursively sort elements before partition and after partition

        sort(arr, low, pi-1);  sort(arr, pi+1, high);

    }

} Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Rest Chat – Avatars, News Feed and State Machine

So, as a next step I decided to implement Avatars, News Feed and State Machine.

Avatars

That was pretty simple to add avatars – add module which will handle image upload/cropping for future upload to the DB Avatar field. I found couple modules to add, but they are not 100% working:

  1. https://github.com/andyshora/angular-image-crop Zoom is not working for chrome 57.0 and angularjs 1.4.8
  2. https://github.com/allenRoyston/ngCroppie is not working at all (I made a bug – https://github.com/allenRoyston/ngCroppie/issues/32).
  3. etc

So, finally I found ngImgCrop module for this purpose.

State Machine

I used classic state machine to handle friendship statuses (and errors), so, I draw a pretty simple scheme with transitions and after that – excel file with description of every transition (including errors handling with proper messages). You will never see this error messages unless you would like to modify existing RESR URLs to try to approve friendship of some unknown user and etc.

Cause this is a pretty simple and well known area – here is some theoretical description and implementation of the state machine:

code snapshot:

case "Restore Subscription":
 switch (relationFriend){
 case 0: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 1: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 10: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 11: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 12: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 20: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 21: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 22: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 23: errorExists = 1;errorText = "There is no ignored friend. "; break;
 case 30: newRelationFriend=21; break;
 } break;
 case "Unsubscribe":

News Feed

In fact, news feed is a somewhat like a messaging, but with the different security and CQL statement of course. Therefore, new requirements for the New Feed are:

  1. Add News Feed item
  2. Edit News Feed item
  3. Delete News Feed item
  4. Add subscription
  5. Delete subscription
  6. Hide News Feed item from everybody
  7. Show News Feed item for friends only
  8. Show News Feed item for everybody
  9. Select favorite authors

I don’t want to make a full description for each requirement and don’t want to describe User Stories because I’m a Facebook user and requirements on this level are pretty obvious and straight forward. I skipped some of these requirements during implementation and will implement them during some of the next iterations.

Let me describe my current database structure (I used draw.io for that):

Let me introduce DB changes:

1)  Introduce new object to store News Feed items

2) Add fields to store avatars

So, as a result I got this demo video and a bunch of items for the next version at my backlog.

Standard
Java, NoSQL, SQL, REST API and other scary words

Angular JS – implement Emoji

Today I’ll show you how to implement custom emoji (in fact – how to replace any text with any image) with AngularJS. In fact, that will kill you browser, cause this is a pretty heaven operation and for the chat application, which refreshes every 3 seconds, such kind of solution is a real disaster.

So, the first step is to implement custom binding for your text, you could easily do it this way:

2017-03-27_12-02-15

As you can see – I’m using emoji function here, and here is my code listing:

var emoticons = {
 ':)' : getUrlToMessages +'img_smile.png',
 ':(' : getUrlToMessages +'img_sad.png',
 ':D' : getUrlToMessages +'img_haha.png',
 ':o' : getUrlToMessages +'img_omg.png'
 }, patterns = [], metachars = /[[\]{}()*+?.\\|^$\-,&#\s]/g;

$scope.emoji = function(message){
   if (message != null){
   // build a regex pattern for each defined property
   for (var i in emoticons) {
     if (emoticons.hasOwnProperty(i)){ // escape metacharacters
     patterns.push('('+i.replace(metachars, "\\$&")+')');
     }
   }
   // build the regular expression and replace
   return message.replace(new RegExp(patterns.join('|'), 'g'), function (match) {
     var escape = typeof emoticons[match] != 'undefined' ? '<img src="' + emoticons[match] + '" />' : match;
     return $sce.trustAsHtml(escape);
   });
 }
}

As far as I don’t want to use any custom images,  I’ll just use decimal code emoji. This is not so straightforward cause Angular’s $sanitize service attempts to convert the characters to their HTML-entity equivalent.  To avoid that HTML going through $sanitize – pass your string through $sce.trustAsHtml:

$scope.emoji = function(message){
 return $sce.trustAsHtml(message);
 }

 

Standard
Java, NoSQL, SQL, REST API and other scary words

Cassandra Datastax and Java – best way to set up connection

I’ll research the best way to make a connection from my Java to Cassandra here. There are a lot of examples how to do that, but the main thing, but I’m developing some kind of chat application on my localhost (will do single insert/update statements, etc.) when all this Spark examples are perfect for analytical workflows.

The first one example is Spark 1.6:

public static JavaSparkContext getCassandraConnector(){
         SparkConf conf = new SparkConf();
         conf.setAppName("Chat");
         conf.set("spark.driver.allowMultipleContexts", "true");
         conf.set("spark.cassandra.connection.host", "127.0.0.1");
         conf.set("spark.rpc.netty.dispatcher.numThreads","2");
         conf.setMaster("local[2]");

         JavaSparkContext sc = new JavaSparkContext(conf);
         return sc;
    }

So, I also got an example for Spark 2.x where the builder will automatically reuse an existing SparkContext if one exists and create a SparkContext if it does not exist. Configuration options set in the builder are automatically propagated over to Spark and Hadoop during I/O. Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Spark Dataset API implementation

Dataframes

Spark introduced Dataframes in Spark 1.3 release. Dataframe overcomes the key challenges that RDDs had.

A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a R/Python Dataframe. Along with Dataframe, Spark also introduced catalyst optimizer, which leverages advanced programming features to build an extensible query optimizer.

Dataframe Features

  • Distributed collection of Row Object: A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database, but with richer optimizations under the hood.
  • Data Processing: Processing structured and unstructured data formats (Avro, CSV, elastic search, and Cassandra) and storage systems (HDFS, HIVE tables, MySQL, etc). It can read and write from all these various datasources.
  • Optimization using catalyst optimizer: It powers both SQL queries and the DataFrame API. Dataframe use catalyst tree transformation framework in four phases,
  • 1.Analyzing a logical plan to resolve references 2.Logical plan optimization 3.Physical planning 4.Code generation to compile parts of the query to Java bytecode.
  • Hive Compatibility: Using Spark SQL, you can run unmodified Hive queries on your existing Hive warehouses. It reuses Hive frontend and MetaStore and gives you full compatibility with existing Hive data, queries, and UDFs.
  • Tungsten: Tungsten provides a physical execution backend whichexplicitly manages memory and dynamically generates bytecode for expression evaluation.
  • Programming Languages supported:
  • Dataframe API is available in Java, Scala, Python, and R.

Continue reading

Standard
Java, NoSQL, SQL, REST API and other scary words

Modification of the SQL ER model to NoSQL Cassandra

Original MySQL ER-model:

mysql

There is no difficulties to implement account management and security case based on UI architecture – there is no need to join tables.

First problem appeared on attempt  to implement friendship data or messages – we need to join users_link and users table for friendship data and we need to join messages and friendship data to filter messages to display here.

There are some shortcuts which will allow us to solve these problems. Continue reading

Standard