[Quick Start] Lucene

Lucene is a full-text search library in Java.It does so by adding content to a full-text index. It then allows you to perform queries on this index, returning results ranked by either the relevance to the query or sorted by an arbitrary field such as a document’s last modified date.The content you add to Lucene can be from various sources, like a SQL/NoSQL database, a filesystem, or even from websites.

Searching & Indexing:

Lucene is fast because it uses indexes for searching instead of searching for the text directly. It uses a Inverted index for searching as it inverts the page centric searching(page->word) to keyword centric searching(word->page). It is similar to searching for keyword related pages at the end of the book instead of searching all pages for the keyword.

Document is the unit of search and index. It consists of one or more fields(key-value pair). Indexing involves adding Documents to an IndexWriter, and searching involves retrieving Documents from an index via an IndexSearcher.

Query:

Querying is done by its own mini language. The Lucene query language allows the user to specify which field(s) to search on, which fields to give more weight to (boosting), the ability to perform boolean queries (AND, OR, NOT) and other functionality.

More on how to use Lucene in programs can be found in tutorialspoint

 

Kernel Panic

Kernel Panic is a term used to denote when a OS cant boot properly or crashed because of a fatal error. This could be because of many reasons like unsupported hardware or missing/corrupted drive or because of unstable updates.It is equivalent of Windows Blue Screen error for UNIX and UNIX based systems.

Kernel Panic sometimes might be caused because of Kernel Oops(In case of error, Kernel will kill the offending process and continue to run). This can be identified with the call trace stack displayed at the time of startup.In these cases, users will be displayed with the grub menu.

Error1: Simplest reason for this panic might be because of less space in /boot folder.Easiest way to fix this issue will be to boot the old kernel from the available list and users can purge old kernels through

sudo apt-get autoremove

and also remove unstable updates.

Error2: “Kernel Panic – not syncing: VFS: Unable to mount root fs on unknown-block(0,0)”
In this case, grub update is needed. Boot old kernel(if available) or use Live CD to boot and run the following commands from any one of the below approach.

Fix 1: Run

dpkg –configure -a

This will update the initramfs hook and fix the broken kernel.
Fix 2: Run

update-initramfs -u -k version

update-grub

Fix 3: This fix can be done when grub is looking in a different path. Find the correct the filesystem where grub is installed using following commands in GRUB prompt(Shift +C during startup),

ls # Should show your drives and partitions hd0,1=sda1
ls (hd0,1)/ # Should include ‘vmlinuz’ and ‘initrd’
ls (hd0,1)/boot/grub # Should see a lot of *.mod files

and then install grub with following commands after booting old kernel/using LiveCD,

sudo mount /dev/sda1 /mnt
sudo grub-install –root-directory=/mnt /dev/sda

Note: 2nd line should have the partition number.

[Quick Scan]Promises – JavaScript

Promises provide a simpler alternative for executing, composing, and managing asynchronous operations when compared to traditional callback-based approaches. They also allow you to handle asynchronous errors using approaches that are similar to synchronous try/catch

Promise represents the result of a asynchronous transactions.  A Promise represents an operation that hasn’t completed yet, but is expected in the future. It allows you to associate handlers to an asynchronous action’s eventual success value or failure reason. This lets asynchronous methods return values like synchronous methods: instead of the final value, the asynchronous method returns a promise of having a value at some point in the future.

A promise is in one of three different states:

• pending – The initial state of a promise.
• fulfilled – The state of a promise representing a successful operation.
• rejected – The state of a promise representing a failed operation.

Once a promise is fulfilled or rejected, it is immutable (i.e. it can never change again).

In short- Helps you write/manage asynchronous calls easily

Brotli – Small Bread for Chrome is coming !

Google Chrome may be one of the fastest browsers around, but that doesn’t mean there isn’t room for improvement. Starting in the very near future, Chrome’s getting Brotli (“small bread” in Swiss German), a new page compression algorithm that decreases load times.

Brotli, which was revealed in September as a replacement for Chrome’s outgoing Zopfli algorithm, uses a more efficient data format to improve the compression of scripts by up to 26 percent. That should mean faster website rendering and better space utilization in most case, but the potential applications extend beyond mere page content. Compression engineer Zoltan Szabadka sees Brotli being used in image optimization and website pre-fetching on unreliable connections, and perhaps even font compression in scenarios where high-resolution typography isn’t a necessity (e.g., smartphones and other small-screen devices). Subsequently, Google expects all users (but especially those on mobile) to see “lower data transfer fees and reduced battery use.”

 

Source: http://google-opensource.blogspot.in/2015/09/introducing-brotli-new-compression.html

Apache Thrift: Networking Stack

Thrift’s networking stack can be represented as follows,

 Transport:

It provides a simple abstraction for reading/writing from/to the network. This enables Thrift to decouple the underlying transport from the rest of the system (like serialization/deserialization).

Protocol:

This specifies how datatypes use the underlying Transport to encode/decode themselves. Thus the protocol implementation governs the encoding scheme and is responsible for (de)serialization. Some examples of protocols in this sense include JSON, XML, plain text, compact binary etc.

Processor:

Service-specific processor implementations are generated by the compiler. The input and output streams are represented by Protocol objects. The Processor essentially reads data from the wire (using the input protocol), delegates processing to the handler (implemented by the user) and writes the response over the wire (using the output protocol).

Server:

A Server pulls together all of the various features described above.

* Create a transport
* Create input/output protocols for the transport
* Create a processor based on the input/output protocols
* Wait for incoming connections and hand them off to the processor

Apache Thrift: Introduction

Thrift is an interface definition language and binary communication protocol that is used to define and create services for numerous languages. It is used as a remote procedure call(RPC) framework and was developed at Facebook for “scalable cross-language services development”.  The Apache Thrift software framework combines a software stack with a code generation engine to build services that work efficiently and seamlessly between almost any language.

 

There’s a lot of repeated work you have to do when you’re writing a server – primarily designing a protocol and writing code to serialize and deserialize messages on the protocol, but also dealing with sockets and managing concurrency, and writing clients in many languages. Thrift automatically does all of this, given a description of the functions you want to expose from your server to clients. It’s also useful for serializing data on disk or into shared memory (where many of the same problems come up).

Windows Tricks: Open Command prompt in any folder

Command Prompt Fans, if you want to open command prompt in any folder,

• Hold down Shift Key and right click on the folder which you want to open in cmd prompt.
• Choose “Open Command Window here” from the context menu that appears.

Windows Tricks: Increase the number of items in Jump Lists

Jump lists are the recent or the pinned items shows when you right click on a particular application in the taskbar.

If you are the person who lean on Jump Items, there may be a time that you have reached the default 10-item in Jump List.

You can increase the number of items in Jump List,

• Right Click on the taskbar and select properties.
• Go to Jump List tab
• Increase the default number of items.

Windows Tricks: Add Locations to Send To menu

• Create Shortcuts to the folders that you would like to add in “Send To” menu.
• Type in “shell:sendto” in windows explorer location bar at the top and press enter.
• You will be taken to a folder. Copy the shortcuts of the folder that you want to add in “Send To” menu.

That’s it ! Right click and check out the new folders added to the menu.

Dive Deep into Oracle DB – Overview of Memory Architecture (2)

The SGA(System Global Area) contains the following data structures:

Database Buffer Cache
The database buffer cache is the portion of the SGA that holds copies of data blocks read from datafiles. All user processes concurrently connected to the instance share access to the database buffer cache. The database buffer cache and the shared SQL cache are logically segmented into multiple sets.

The buffers in the cache are organized in two lists: the write list and the least recently used (LRU) list. The write list holds dirty buffers, which contain data that has been modified but has not yet been written to disk. The LRU list holds free buffers, pinned buffers, and dirty buffers that have not yet been moved to the write list. Free buffers do not contain any useful data and are available for use. Pinned buffers are currently being accessed.

When an Oracle process accesses a buffer, the process moves the buffer to the most recently used (MRU) end of the LRU list. As more buffers are continually moved to the MRU end of the LRU list, dirty buffers age toward the LRU end of the LRU list.

When the user process is performing a full table scan, it reads the blocks of the table into buffers and puts them on the LRU end (instead of the MRU end) of the LRU list. This is because a fully scanned table usually is needed only briefly, so the blocks should be moved out quickly to leave more frequently used blocks in the cache.

Redo Log Buffer

The redo log buffer is a circular buffer in the SGA that holds information about changes made to the database. This information is stored in redo entries. Redo entries contain the information necessary to reconstruct, or redo, changes made to the database by INSERT, UPDATE, DELETE, CREATE, ALTER, or DROP operations. Redo entries are used for database recovery, if necessary.

Redo entries are copied by Oracle database processes from the user’s memory space to the redo log buffer in the SGA. The redo entries take up continuous, sequential space in the buffer. The background process LGWR writes the redo log buffer to the active redo log file (or group of files) on disk.

Shared Pool

The shared pool portion of the SGA contains the library cache, the dictionary cache, buffers for parallel execution messages, and control structures.
The library cache includes the shared SQL areas, private SQL areas (in the case of a shared server configuration), PL/SQL procedures and packages, and control structures such as locks and library cache handles.

The data dictionary is a collection of database tables and views containing reference information about the database, its structures, and its users. Oracle accesses the data dictionary frequently during SQL statement parsing. This is stored in Dictionary Cache.

Large Pool

The database administrator can configure an optional memory area called the large pool to provide large memory allocations for:

• Session memory for the shared server and the Oracle XA interface (used where transactions interact with more than one database)
• I/O server processes
• Oracle backup and restore operations

Java Pool

Java pool memory is used in server memory for all session-specific Java code and data within the JVM. Java pool memory is used in different ways, depending on what mode the Oracle server is running in.
The Java Pool Advisor statistics provide information about library cache memory used for Java and predict how changes in the size of the Java pool can affect the parse rate.

PGA(Program Global Area) contains the following,

Private SQL Area

A private SQL area contains data such as bind information and runtime memory structures. Each session that issues a SQL statement has a private SQL area. Each user that submits the same SQL statement has his or her own private SQL area that uses a single shared SQL area. Thus, many private SQL areas can be associated with the same shared SQL area.

The private SQL area of a cursor is itself divided into two areas whose lifetimes are different:

• The persistent area, which contains, for example, bind information. It is freed only when the cursor is closed.
• The run-time area, which is freed when the execution is terminated.

Session Memory

Session memory is the memory allocated to hold a session’s variables (logon information) and other information related to the session. For a shared server, the session memory is shared and not private.