Configuring Virtual Machines to connect Internet

Platform for VMs : Virtual box

Steps:

  • Select the Virtual Machine on the left pane and then open the settings.
  • Go to the Network option in left pane of the settings dialog box.
  • In the Center pane , select the network adapter “Attached to ” field as “Bridged Adapter” and then choose your network driver in the dropdown box below.
  • Then set the “Promiscuous Mode” to “Allow All”
  • Now save and start the VM.
  • Then in the terminal, check whether eth0 port is up or not by

ifup eth0

  • Now you test your internet connection by pinging in the terminal.
Advertisements

Storm supervisor not registering itself with the zookeeper

Problems:

1.) When supervisor is in other node than that of the zookeeper,

ConnectionState [ERROR] Connection timed out
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss

2.) When supervisor is in the same node as of the zookeeper,

supervisor [ERROR] Error on initialization of server mk-supervisor
java.net.UnknownHostException: AI: AI

 

Solution:

This is because the firewall of the zookeeper system is not allowing it to accept the connections.While we can solve the 2nd scenario by adding the ip address and the hostname to the /etc/hosts file. Both the problems can be fixed by turning the firewall off by following commands,

service iptables save

service iptables stop

Virtual Box – Pae not present

Problem:

The Kernel requires the following features on the CPU: pae

Solution:

  • Open the Virtual Box Manager, select the system on the left side and click on Settings in the top.
  • When the settings dialog is opened, click on the system tab on the left and select the processor tab
  • Then in the Extended Features panel, check the Enable PAE/NX checkbox. Now save the settings and start the VM.

Learn Haskell

Haskell is a purely functional programming language. In imperative languages you get things done by giving the computer a sequence of tasks and then it executes them. While executing them, it can change state. In purely functional programming you don’t tell the computer what to do as such but rather you tell it what stuff is. The factorial of a number is the product of all the numbers from 1 to that number, the sum of a list of numbers is the first number plus the sum of all the other numbers, and so on. You express that in the form of functions. In purely functional languages, a function has no side-effects. The only thing a function can do is calculate something and return it as a result. At first, this seems kind of limiting but it actually has some very nice consequences: if a function is called twice with the same parameters, it’s guaranteed to return the same result. That’s called referential transparency and not only does it allow the compiler to reason about the program’s behavior, but it also allows you to easily deduce (and even prove) that a function is correct and then build more complex functions by gluing simple functions together.

Haskell is lazy. That means that unless specifically told otherwise, Haskell won’t execute functions and calculate things until it’s really forced to show you a result. That goes well with referential transparency and it allows you to think of programs as a series of transformations on data. It also allows cool things such as infinite data structures. Say you have an immutable list of numbers xs = [1,2,3,4,5,6,7,8] and a function doubleMe which multiplies every element by 2 and then returns a new list. If we wanted to multiply our list by 8 in an imperative language and did doubleMe(doubleMe(doubleMe(xs))), it would probably pass through the list once and make a copy and then return it. Then it would pass through the list another two times and return the result. In a lazy language, calling doubleMe on a list without forcing it to show you the result ends up in the program sort of telling you “Yeah yeah, I’ll do it later!”. But once you want to see the result, the first doubleMe tells the second one it wants the result, now! The second one says that to the third one and the third one reluctantly gives back a doubled 1, which is a 2. The second one receives that and gives back 4 to the first one. The first one sees that and tells you the first element is 8. So it only does one pass through the list and only when you really need it. That way when you want something from a lazy language you can just take some initial data and efficiently transform and mend it so it resembles what you want at the end.

Haskell is statically typed. When you compile your program, the compiler knows which piece of code is a number, which is a string and so on. That means that a lot of possible errors are caught at compile time. If you try to add together a number and a string, the compiler will whine at you. Haskell uses a very good type system that has type inference. That means that you don’t have to explicitly label every piece of code with a type because the type system can intelligently figure out a lot about it. If you say a = 5 + 4, you don’t have to tell Haskell that a is a number, it can figure that out by itself. Type inference also allows your code to be more general. If a function you make takes two parameters and adds them together and you don’t explicitly state their type, the function will work on any two parameters that act like numbers.

Haskell is elegant and concise. Because it uses a lot of high level concepts, Haskell programs are usually shorter than their imperative equivalents. And shorter programs are easier to maintain than longer ones and have less bugs.

Source:  http://learnyouahaskell.com

Facebook open-sources Haxl: Big Data Haskell library

Facebook announced the open-sourcing of Haxl, a Haskell library used to greatly simplify access to remote data, such as databases and API web-based services. This release continues a trend of Facebook’s strong bent towards data science and the processing of complex datasets, and at the same time sustains a commitment to the open source community.

The Haxl project (the combination of Haskell + FXL) at Facebook provides an abstraction layer to access the various pieces of content at the ready. Haxl is as an embedded domain-specific language implemented in Haskell. In other words, Haxl is implemented in a Haskell framework that is used by application programmers to develop Haskell programs that access the various Facebook components via a uniform API.

The Haxl project is the brainchild of former Glasgow Haskell Compiler lead Simon Marlow. Haxl provides clean coding mechanisms for writing business logic (filter rules, queries, etc.) that act on remote data. Along with the announcement, Facebook posted a blog article that outlines Haxl’s capabilities and the logic behind its implementation for developers to understand.

As an example, Facebook currently uses FXL (Haxl’s “context parser” parent) as part of a spam and malware detection system. FXL allows for the construction of context-sensitive rules that can interrelate and return answers based on a set of facts. Such as “Is this URL associated with malware?” where FXL responds with “Yes, block this” or “No, carry on.”

With Haxl, developers get the power and reliability of the Haskell programming language with a library that allows them to access the context and interrelation strengths of FXL.

Facebook’s Big Data history
As a social media company, Facebook is swimming in an ocean of user generated data. Every post, image, interaction, and like on Facebook is a data point and with 1.3 billion monthly active users (as of January 2014) that’s a lot of interrelations. The result: Facebook has a strong history with Big Data and Haxl yet another milestone.

In her article about Facebook’s Big Data story, Kristen Nicole mentions that Facebook is a prime target for Big Data solutions. Solutions such as Facebook’s specialized search strategy: the Graph Search. The Graph Search is another invention of Facebook designed to allow for filtering a group of users by specified criteria contextualized by relationships between users.

Haxl fits nicely into these other developments from Facebook in that it’s a library that abstracts away the filtering process of contextualizing interrelations in datasets.

Facebooks commitment to open source and developers
By open sourcing Haxl, Facebook is essentially letting it loose into a broader community of developers. As a result, developers will be able to work on their own repo of Haxl, see the github page for code and documentation, and implement it in their own software.

Facebook has been a strong proponent of the open source community and its benefits. Haxl is being added to growing stable of Facebook released open source projects.

JSON with Python

The demjson package is a comprehensive Python language library to read and write JSON, the popular language-independent data format standard.It includes a command tool, jsonlint, that allows you to easily check and validate any JSON document, and spot any potential data portability issues. It can also reformat and re-indent a JSON document to make it easier to read.

 

Installing demjson:

Download and install the package to start encoding and decoding JSON in python with the following commands,

$tar xvfz demjson-1.6.tar.gz
$cd demjson-1.6
$python setup.py install

 

Eg:

 >>> import demjson

>>> demjson.encode( [‘one’,42,True,None] )   # From Python to JSON

‘[“one”,42,true,null]’

>>> demjson.decode( ‘[“one”,42,true,null]’ ) # From JSON to Python

[‘one’, 42, True, None]

JSON with Php

The common functions of the JSON extension in PHP by default are as follows,

 

json_encode() – Encodes the value into JSON format

Eg:

$arr = array(‘a’ => 1, ‘b’ => 2, ‘c’ => 3, ‘d’ => 4, ‘e’ => 5);

echo json_encode($arr);

 

json_decode() – Decodes tha value from the JSON format

Eg:

$json = ‘{“foo-bar”: 12345}’;

$obj = json_decode($json);

print $obj->{‘foo-bar’};

 

json_last_error() – Returns the last error occurred

Eg:

    switch (json_last_error()) {

case JSON_ERROR_NONE:

echo ‘ – No errors’;

break;

case JSON_ERROR_DEPTH:

echo ‘ – Maximum stack depth exceeded’;

break;

case JSON_ERROR_STATE_MISMATCH:

echo ‘ – Underflow or the modes mismatch’;

break;

case JSON_ERROR_CTRL_CHAR:

echo ‘ – Unexpected control character found’;

break;

case JSON_ERROR_SYNTAX:

echo ‘ – Syntax error, malformed JSON’;

break;

case JSON_ERROR_UTF8:

echo ‘ – Malformed UTF-8 characters, possibly incorrectly encoded’;

break;

default:

echo ‘ – Unknown error’;

break;

}