Spring Batch - simple Range Partitioner

This sample range partitioner could be used in Spring Batch application to partition data based on the Grid size and allocate them to individual executors.

[java]

package com.hashfold.spring.batch;

import java.util.Collections;
import java.util.HashMap;
import java.util.Map;

import org.springframework.batch.core.partition.support.Partitioner;
import org.springframework.batch.item.ExecutionContext;

public class RangePartitioner implements Partitioner {

private long start;
private long end;

public final void setStart(long start) {
this.start = start;
}

public final void setEnd(long end) {
this.end = end;
}

public RangePartitioner() {
}

@Override
public Map<String, ExecutionContext> partition(int gridSize) {

long rangeSize = end - start;
if (rangeSize <= 0)
return Collections.<String, ExecutionContext> emptyMap();

int numberOfIntervals = gridSize;
long sizeOfSmallSublists = rangeSize / numberOfIntervals;
long sizeOfLargeSublists = sizeOfSmallSublists + 1;
long numberOfLargeSublists = rangeSize % numberOfIntervals;
long numberOfSmallSublists = numberOfIntervals - numberOfLargeSublists;

Map<String, ExecutionContext> result = new HashMap<String, ExecutionContext>();

long numberOfElementsHandled = 0;

for (long i = 0; i < numberOfIntervals; i++) {

long size = i < numberOfSmallSublists ? sizeOfSmallSublists
: sizeOfLargeSublists;

long threadSeq = i + 1;
long startId = numberOfElementsHandled;
long endId = numberOfElementsHandled + size;

/*
* happens when range is less than the grid size. e.g. start=0, end
* = 2 and gridSize=3
*/
if ((endId - startId) <= 0)
continue;

ExecutionContext value = new ExecutionContext();

value.putLong("start", startId);

value.putLong("end", endId);

value.putString("name", "Thread-" + threadSeq);
result.put("partition" + threadSeq, value);

System.out.println("Starting : Thread-" + threadSeq + " : "
+ startId + " - " + endId);

numberOfElementsHandled += size;
}

return result;
}

// used only to test some cases. we should use junit and remove this method.
public static void main(String args[]) {
RangePartitioner rp = new RangePartitioner();
rp.setStart(1);
rp.setEnd(11);
rp.partition(3);

}

}

[/java]

Batch configuration change

[xml]

<!-- partitioner job -->
<job id="partitionJob" xmlns="http://www.springframework.org/schema/batch">

<!-- master step, 10 threads (grid-size) -->
<step id="masterStep">
<partition step="slave" partitioner="transactionPartitioner">
<handler grid-size="10" task-executor="taskExecutor" />
</partition>
</step>
</job>

<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="10" />
<property name="maxPoolSize" value="10" />
</bean>

<bean id="transactionPartitioner" class="com.hashfold.spring.batch.RangePartitioner">
<!-- we should process all records with range from [start, end). Note end is not included. it processes upto end-1-->
<property name="start" value="1" /> <!-- this number is included-->
<property name="end" value="11" /> <!-- this number is excluded-->
<!-- DB query should be executed with "WHERE ID >= :start and ID < :end" clause-->
</bean>

[/xml]

SQL Operators order by the performance

 

key operators used in the WHERE clause, ordered by their performance. Those operators at the top will produce results faster than those listed at the bottom.

=
>
>=
<
<=
LIKE
<>

 

via here

GIT Video Tutorials - learn Git from experts

GIT-SCM.com tutorials:


GIT Casts:


GIT 101:


GIT & GitHub:


text book @ http://git-scm.com/book

git flow: http://nvie.com/posts/a-successful-git-branching-model/

using git flow: http://yakiloo.com/getting-started-git-flow/

Improving performance of Java String methods

Regular expression matching always comes with the cost of pattern matching. In case the pattern is not precompiled then the the expression parsing happens every time the code is invoked.

Some of Java String methods use regular expressions e.g. matches(...), replaceFirst(...), replaceAll(...) and split(...). These methods actually use Pattern matching library methods internally however these patterns are parsed every time these methods are invoked. The performance impact is significant if these methods are called too frequently or in high traffic zone.

Java library provides Pattern package which could be used to precompile the regular expressions.

Here are equivalent code which uses the precompiled pattern.

1) String.split(regex)

[java]

private static final String regex = "\\.";

String[] keys = str.split(regex);

[/java]

 

[java]
import java.util.regex.Pattern;
private static final Pattern myPattern = Pattern.compile(regex);
String[] keys = myPattern.split(str);

[/java]

2) String.replace*(regex...)

2.1) replaceFirst() - http://docs.oracle.com/javase/6/docs/api/java/lang/String.html#replaceFirst(java.lang.String, java.lang.String)

[java]

str.replaceFirst(regex, repl) yields exactly the same result as the expression
"Pattern.compile(regex).matcher(str).replaceFirst(repl)""

[/java]

 

[java]

private static final String regex = "\\.";
private static final Pattern myPattern = Pattern.compile(regex);
myPattern.matcher(str).replaceFirst(repl);

[/java]

 

2.2) replaceAll() - http://docs.oracle.com/javase/6/docs/api/java/lang/String.html#replaceAll(java.lang.String, java.lang.String)

[java]

str.replaceAll(regex, repl) yields exactly the same result as the expression
"Pattern.compile(regex).matcher(str).replaceAll(repl)""

[/java]

 

[java]

private static final String regex = "\\.";
private static final Pattern myPattern = Pattern.compile(regex);
myPattern.matcher(str).replaceAll(repl);

[/java]

3) matches(regex) - http://docs.oracle.com/javase/6/docs/api/java/lang/String.html#matches(java.lang.String)

[java]

str.matches(regex) yields exactly the same result as the expression
"Pattern.matches(regex, str)"

[/java]

 

[java]
private static final String regex = "\\.";
private static final Pattern myPattern = Pattern.compile(regex);
myPattern.matches(regex, str);

[/java]

The Zen of Python, by Tim Peters (python -m this)

$ python -m this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Introducing Google Official Blog Reader

Its a simple Google blog news reader. Written in JavaScript, it uses Google's feed history system to pull feed entries and shows the latest news.

Try it here. http://www.hashfold.com/news/

Note: It uses user's internet to pull the news updates.

send me your feedback and ideas to improve the performance and look n feel of the page.

Writing Your First DSL using Groovy

Writing Your First DSL using Groovy
In this example, we will be writing a DSL to configure the valid values for the columns of a given table.

Below is how we write the DSL:
Sample.groovy:

[java]
package com.hashfold.groovy.dsl

import com.hashfold.groovy.dsl.Schema

Schema.create("MyTable") {
column1 1,2,3,4,5,6,7,8,9,10
column2 50,80,{println 90+1 }
column3 "fixed", { param -> println "envData=${envData}-param="+param }
}

[/java]

Schema.groovy:

[java]
package com.hashfold.groovy.dsl

import groovy.xml.MarkupBuilder

/**
* Processes a simple DSL to create various formats of a memo: xml, html, and text
*/
class Schema {

String name
//def columns = []
def cols = [:]

/**
* This method accepts a closure which is essentially the DSL. Delegate the closure methods to
* the DSL class so the calls can be processed
*/
def static Object create(String dataName, closure) {
Schema dataDsl = new Schema()
dataDsl.name = dataName

//println dataDsl.name

// any method called in closure will be delegated to the memoDsl class
closure.delegate = dataDsl
closure()

//test closure will be invoked from Java!
//dataDsl.cols["testClosure"] = [{ param -> println "print my closure - "+param }]

return dataDsl.cols
}

def storage = [:]
def propertyMissing(String name, value) {
println "Undefined Property setter: ${name} = ${value}"
storage[name] = value
}
def propertyMissing(String name) {
println "Undefined Property getter: ${name}"
storage[name]
}

/*
* This method will be called for each column names e.g. column1…
*/
def methodMissing(String methodName, args) {

List data = []

args.each {

if(it instanceof Closure) {
/*
* lets not evaluate the 'validate' closure.
* we leave it to evaluate on Java side against
* actual data. in case the evaluation returns 0/False,
* we throw validation error
*/
if(methodName.toLowerCase() == "validate") {
data << it
}
else {
//lets not evaluate the closure here. leave it for java
//data << it()
data << it
}
}
else {
data << it } } //this is the last statement of the method which will be returned from the Groovy to Java cols[methodName] = data } //This is the test method which executes the given closure def static test(String name, closure) { println name Schema dataDsl = new Schema() closure.delegate = dataDsl closure() } def getDump() { println ">>>Dumping..."
println name
cols.each {
println "name:"+it.name
println "obj:"+it.range

println "\t"+it.range
it.range.each {
println "\t\t"+it
if(it instanceof Closure) {
it()
}
}
}
}

}
[/java]

Here is how we call the above Groovy script from Java and execute it:

GroovyCaller.java:

[java]
import groovy.lang.Binding;
import groovy.lang.GroovyShell;
import java.io.File;

public class GroovyCaller {

public static void main(String[] args) throws Exception {

Binding binding = new Binding();
//Lets pass some data from Java to Groovy using Binding
binding.setVariable("envData", new Integer(2));
GroovyShell shell = new GroovyShell(binding);

File file = new File("Sample.groovy");

Object value = shell.evaluate(file);
//Lets print the data returned by Groovy (from last method call of MethodMissing)
System.out.println(value);


/* lets invoke the Closure if any! Note here that
* any column value written inside brackets ‘{}’ will be treated
* as Groovy Closure which are execution blocks
*/
LinkedHashMap<String, ArrayList<Object>> cols = (LinkedHashMap<String, ArrayList<Object>>) value;

for (String variable : cols.keySet()) {

ArrayList<Object> list = cols.get(variable);
for(Object data: list) {
if(data instanceof Closure) {
Closure c = (Closure) data;
c.call("MyTest");
}
}
}
}

}
[/java]

Output from GroovyCaller.java:

{column1=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
column2=[50, 80, com.hashfold.groovy.dsl.Sample$_run_closure1_closure2@37963796],
column3=[fixed, com.hashfold.groovy.dsl.Sample$_run_closure1_closure3@1d001d0],
91
envData=3 - param=MyTest

The above last two lines are the result of the Closure execution from Java.


In order to build the java code, you need to add below plugin to your maven pom.xml file:

[xml]
<plugin>
<groupId>org.codehaus.gmaven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.2</version>
<configuration>
<providerSelection>1.7</providerSelection>
</configuration>
<dependencies>
<dependency>
<groupId>org.codehaus.gmaven.runtime</groupId>
<artifactId>gmaven-runtime-1.7</artifactId>
<version>1.2</version>
<exclusions>
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>1.7.0</version>
</dependency>
</dependencies>
<executions>
<execution>
<goals>
<goal>generateStubs</goal>
<goal>compile</goal>
<goal>generateTestStubs</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>

</configuration>
</plugin>
[/xml]

Also add below dependencies:

[xml]
<dependencies>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>1.7.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.1</version>
</dependency>
</dependencies>
[/xml]

Groovy DSL Reference Book: Groovy for Domain-Specific Languages

Automating Virtual Machine creation using VirtualBox, Vagrant and Puppet

Introduction

In this tutorial, I'll be going through the process of creating Virtual Machine using VirtualBox, Vagrant and Puppet. Lets first understand what these mean and then we can go through the manual process of creating Virtual Machines and then automating everything using Vagrant and Puppet.

VirtualBox

VirtualBox is a general-purpose full virtualizer for x86 hardware, targeted at server, desktop and embedded use.

site: https://www.virtualbox.org/

Documentation: https://www.virtualbox.org/manual/

Commands: https://www.virtualbox.org/manual/ch08.html#idp5692800

Vagrant

Vagrant is a tool for building complete development environments. With an easy-to-use workflow and focus on automation, Vagrant lowers development environment setup time, increases development/production parity, and makes the "works on my machine" excuse a relic of the past.

Vagrant creates and manages the Virtual Machine based on VirtualBox. Internally it uses VirtualBox commands for the management.

Vagrant supports several provisioners using which we can install and configure any software on the Virtual Machine. Some of the provisioners are:

• Chef Solo


• Chef Server


• Puppet Standalone


• Puppet Server


• Shell

The most popular one is Puppet which we will be using it to provision our Virtual Machine.

Site: http://www.vagrantup.com

Documentation: http://docs.vagrantup.com/v1/docs/

Commands: http://docs.vagrantup.com/v1/docs/commands.html

Free Vagrant Boxes: http://www.vagrantbox.es/

Puppet

Puppet is IT automation software that helps system administrators manage infrastructure throughout its lifecycle, from provisioning and configuration to patch management and compliance. Using Puppet, you can easily automate repetitive tasks, quickly deploy critical applications, and proactively manage change, scaling from 10s of servers to 1000s, on-premise or in the cloud.

Puppet is available as both open source and commercial software. You can see the differences here and decide which is right for your organization.

Puppet can be configured to work in Standalone and Master Slave modes.

Site: https://puppetlabs.com/puppet/what-is-puppet/

Syntax: http://www.puppetcookbook.com/

Automating the Virtual Machine Creation Process

Assumption: Hope you have already installed VirtualBox and Vagrant on your Host Machine(machine on which you are going to create the Virtual Machine).

Note that the Virtual Machine is called Guest Machine because its running on your Host Machine.

1. Pick your base box from http://www.vagrantbox.es/

For our example, I picked http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.3-x86_64-v20130101.box  which is ‘CentOS 6.3 x86_64 Minimal (VirtualBox Guest Additions 4.2.6, Chef 10.16.4, Puppet 3.0.2)’.

These free boxes are configured with all the essential softwares e.g. Guest Additions, Sudo access, SSH, Puppet etc and also have under account vagrant/vagrant created.

These boxes have root/admin and vagrant/vagrant accounts already created.

2. In case you do not want to create a Box based on the public boxes then you can do below:

2.1.       Create Virtual Machine on VirtualBox. Lets say you named it as myDevBox.

2.2.       Follow steps @ http://docs.vagrantup.com/v1/docs/base_boxes.html

Make sure you install necessary softwares e.g. Puppet, SSH, Guest Additions, Ruby & RubyGems. Also create below user account:

Hostname: vagrant-[os-name], e.g. vagrant-debian-lenny
Domain: vagrantup.com
Root Password: vagrant
Main account login: vagrant
Main account password: vagrant

Make sure you also create root/admin account.

We will be following step 2.1 for our example.

3. Enable the logging. It helps to debug the issue if any.

export VAGRANT_LOG=info (or set VAGRANT_LOG=info in case you are running on windows)

4. Adding box to our environment

This will start downloading the .box file (these files are generally in GB).

vagrant box add myDevBox http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.3-x86_64-v20130101.box

This process also creates a file named ‘Vagrantfile’ which holds the configuration for the Virtual Machine.

5. Initialize the VirtualBox with the box we added above

vagrant init myDevBox

6. Start the VirtualMachine

vagrant up

7. Logging into the VirtualMachine

7.1.       Using SSH via vagrant

vagrant ssh

7.2.       Using SSH via port forwarding

Port forwarding can be done using VirtualBox GUI or running command from http://www.virtualbox.org/manual/ch06.html#natforward

ssh -p 2222 root@127.0.0.1

7.3.       Start Virtual Machine from VirtualBox GUI directly and work inside the Guest Machine itself

8. Once you logged into the Virtual Machine, you can work on it. Install additional softwares, configure it based on your requirements.

9. Exporting/Packaging the current working Virtual Machine and distributing it to other developers

vagrant package --include README.txt --vagrantfile Vagrantfile --base MyDevBox --output MyDevBox.box

Note: --include README.txt :  means you could attach additional files to the .box file. This along with Vagrantfile will be created once someone starts utilizing it (step #4).

The box will be created with MyDevBox.box name which you can distribute to others

Automating Essential Software Installation and Configuration Process

Anything we do at Step#8 could be automated using Puppet. All you need to do is put the puppet configuration inside Vagrantfile and run “vagrant up” or only “vagrant provision”.

So far in above section we followed the commands to download box, port forwarding, directory sharing etc. All these can be done by configuring the Vagrantfile itself.

Below is the sample Vagrantfile :

[ruby]

# Author: HashFold

# description: created VM and provisions basic tools

Vagrant::Config.run do |config|

config.vm.box = "MyDevBox"

config.vm.box_url = "http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.3-x86_64-v20130101.box "

config.vm.host_name = "com.mycompany.virtualmachines.centos.project1"

#--- common ports and share folders ---

#ssh -p 2222 root@127.0.0.1

config.vm.forward_port 22, 2222, :adapter => 1

#config.vm.network :hostonly, "33.33.33.10", :adapter => 1

#--- Directory sharing it shares /myshare (inside Guest Machine) to ~/myshare (on Host Machine, note that ~ mapped to corresponding user’s home directory)

config.vm.share_folder "my-data-share", "/myshare", File.expand_path('~/myshare'), :create => true

#sharing current folder makes it OS independent mount

config.vm.share_folder "v-root", "/vagrant", File.expand_path('~/vagrant'), :create => true

#--- common ports for lamp system ---

#accounts: root/admin, vagrant/vagrant

#mysql passwd is blank. (Or “admin”)

#mysql client port

config.vm.forward_port 3306, 3306, :adapter => 1

#httpd/apache @ http://127.0.0.1:8080/

config.vm.forward_port 80, 8080

#--- common ports for oracle xe ---

#ORACLE XE: system/manager@http://localhost:8081/

config.vm.forward_port 1521, 1521, :adapter => 1

config.vm.forward_port 8081, 8081, :adapter => 1

#--- Puppet configuration ---

#--- suppose puppet_store directory is in the current folder ---

config.vm.provision :puppet do |puppet|

puppet.module_path = "puppet_store/modules"

puppet.manifests_path = "puppet_store/manifests"

puppet.manifest_file  = "mybox.pp"

end

#--- virtual machine configurational parameters

#get list of VMBox commands @http://www.virtualbox.org/manual/ch08.html#vboxmanage-modifyvm

Vagrant::Config.run do |config|

config.vm.customize [

"modifyvm", :id,

"--cpus", 2,

"--memory", 2048,

"--name", "MyDevBox"

]

end

end

[/ruby]

Lets look inside of puppet_store and see how the files are laid out:

$ find puppet_store

[ruby]

puppet_store

puppet_store/manifests

<b>puppet_store/manifests/</b><b>mybox</b><b>.pp</b><b>  --main driver for puppet</b>

puppet_store/modules

puppet_store/modules/commontools

puppet_store/modules/commontools/files

puppet_store/modules/commontools/files/epel_ex.repo

puppet_store/modules/commontools/files/populate_shared_data.sh

puppet_store/modules/commontools/files/puppet.repo

puppet_store/modules/commontools/files/ruby.repo

puppet_store/modules/commontools/files/rubyworks.repo

puppet_store/modules/commontools/manifests

puppet_store/modules/commontools/manifests/init.pp <b>– 2nd main driver (called from mybox.pp)</b>

puppet_store/modules/lamp

puppet_store/modules/lamp/files

puppet_store/modules/lamp/files/apache2

puppet_store/modules/lamp/files/apache2/apache2.CentOS.conf

puppet_store/modules/lamp/files/apache2/apache2.conf

puppet_store/modules/lamp/files/apache2/apache2.Ubuntu.conf

puppet_store/modules/lamp/files/mysql

puppet_store/modules/lamp/files/mysql/my.CentOS.cnf

puppet_store/modules/lamp/files/mysql/my.cnf

puppet_store/modules/lamp/files/mysql/my.Ubuntu.cnf

puppet_store/modules/lamp/files/mysql/mysql_create_user.sh

puppet_store/modules/lamp/files/mysql/mysql_create_user.sql

puppet_store/modules/lamp/files/mysql/mysql_import_data.sh

puppet_store/modules/lamp/files/sysconfig

puppet_store/modules/lamp/files/sysconfig/httpd.CentOS

puppet_store/modules/lamp/files/sysconfig/httpd.Ubuntu

puppet_store/modules/lamp/files/www

puppet_store/modules/lamp/files/www/index.php

puppet_store/modules/lamp/manifests

puppet_store/modules/lamp/manifests/init.pp<b> – 3rd main driver for LAMP system installtion</b>

puppet_store/modules/oracle

puppet_store/modules/oracle/files

puppet_store/modules/oracle/files/oracle-shm

puppet_store/modules/oracle/files/oracle-xe-11.2.0-1.0.x86_64.rpm.zip

puppet_store/modules/oracle/files/oracle_create_user.sh

puppet_store/modules/oracle/files/oracle_create_user.sql

puppet_store/modules/oracle/files/oracle_create_user_data_import.sh

puppet_store/modules/oracle/files/oracle_db.dmp.zip

puppet_store/modules/oracle/files/oracle_path.sh

puppet_store/modules/oracle/files/xe-sysctl.conf

puppet_store/modules/oracle/files/xe.rsp

puppet_store/modules/oracle/manifests

puppet_store/modules/oracle/manifests/init.pp <b>– 4th main driver for LAMP system installtion</b>

[/ruby]

1^st Level Puppet file:

The main driver for puppet is puppet_store/manifests/mybox.pp  :

[ruby]

# Project: Project1

# Author: HashFold

# description: main driver script for Puppet

# read more puppet syntax @ http://www.puppetcookbook.com/

#make sure the puppet is installed on the .box otherwise rest of the puppet provisioning will not function

group { "puppet":

ensure => "present",

}

Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

#below is to fix "yum.Errors.RepoError: database disk image is malformed" error

exec {

"common_tool_set_yum_clean":

alias => "common_tool_set_yum_clean",

command => "sudo yum -y clean  dbcache",

path => '/usr/bin:/bin:/usr/sbin:/sbin',

cwd => "/tmp",

user => root;

}

File { owner => 0, group => 0, mode => 0644 }

file { '/etc/motd':

content => "Welcome to My Developer Box!

Managed by HashFold team.

contact Admin at My Company.com for any issues.\n"

}

node default {

include commontools::install

include lamp

include oracle::server

include oracle::xe

}

[/ruby]

2^nd Level Puppet file:

puppet_store/modules/commontools/manifests/init.pp – 2nd main driver (called from mybox.pp)

[ruby]

# Project: Project1

# Author: HashFold

# description: runs common commands

class commontools::install {

file {

"/etc/yum.repos.d/epel_ex.repo":

source => "puppet:///modules/commontools/epel_ex.repo";

"/etc/yum.repos.d/puppet.repo":

source => "puppet:///modules/commontools/puppet.repo";

"/etc/yum.repos.d/ruby.repo":

source => "puppet:///modules/commontools/ruby.repo";

"/etc/yum.repos.d/rubyworks.repo":

source => "puppet:///modules/commontools/rubyworks.repo";

"/tmp/populate_shared_data.sh":

mode => 0755,

source => "puppet:///modules/commontools/populate_shared_data.sh";

}

exec { 'populate_shared_data':

command => "/usr/bin/sudo /tmp/populate_shared_data.sh",

require => [File["/tmp/populate_shared_data.sh"]];

}

}
<pre>[/ruby]

In populate_shared_data.sh, you could put any commands which you would want to run at the time of Virtual Machine setup by vagrant (vagrant provision).
One example could be to run below command to download mysql installer file:
sudo wget http://localhost/mysql-workbench-gpl-5.2.42-win32.msi (this is not for Centos however this will be visible under user’s shared folder so they can click and install these free softwares.)

Some trouble shooting tips

1.     Provisioner error:

[ruby]</pre>
←[0;36mnotice: /Stage[main]/Oracle::Server/Exec[sudo yum -y install libaio bc flex]/returns: executed successfully←[0m

←[0;36mnotice: /Stage[main]/Oracle::Xe/Exec[unzip xe]/returns: executed successfully←[0m

←[1;35merr: /Stage[main]/Oracle::Xe/Package[oracle-xe]/ensure: change from absent to latest failed: Could not update: Execution of '/bin/rpm -i --oldpackage /tmp/Disk1/oracle-xe-11.2.0-1.0.x86_64.rpm' returned 1:

This system does not meet the minimum requirements for swap space.  Based on

the amount of physical memory available on the system, Oracle Database 11g

Express Edition requires <b>2004 MB</b> of swap space. This system has <b>703 MB</b>

of swap space.  Configure more swap space on the system and retry the installation.

error: %pre(oracle-xe-11.2.0-1.0.x86_64) scriptlet failed, exit status 1

error:   install: %pre scriptlet failed (2), skipping oracle-xe-11.2.0-1.0

[/ruby]

Solution:

Log into the Virtual Machine and run below commands:

[ruby]

swapoff /dev/mapper/VolGroup00-LogVol01

dd if=/dev/zero of=/home/swapfile bs=1024 count=2097152

mkswap /home/swapfile

swapon /home/swapfile

swapon -a

cp /etc/fstab /etc/fstab.backup_`date +%N`

'/home/swapfile swap swap defaults 0 0' >> /etc/fstab

swapon –s

[/ruby]

comparing ebay and amazon home pages using 3D layout structures

eBay.com view:

amazon.com view:

ebay: category section floats outside of the view area and rest of the layout area looks perfectly fine.

amazon: no floating elements found except the layout width is uneven.

Facebook Graph Search - some interesting queries

today I got approved by Facebook to use their graph search.

I played with it a bit and found it very useful in exploring the unknown/unexplored.

here are few of them:

1. Friends of friends who like to dance



2. Friends of my friends who live in San Francisco, California and work at Google



3.  This time, I'm looking for anyone who love playing cricket but is from my city. (I'm big fan of cricket so need to find more players)

"People who like Cricket and live in San Jose, California"



4. looking for job and willing to apply to company 'A'? you need to explore 1st, 2nd degree of connections to work in that company

"Friends of my friends who work at Google "



5. Facebook doesn't understand what is what - they seems to be missing natural language processing and I hope they know it and are working to improve it.

 



 

Graph Search if used cleverly, it could provide lots of insite into what we need.

 

that's all folks!

 

 

Amazing Page Architecture - Measuring the thickness of the web pages

disclaimer: i'm not css/html expert.

I believe that browsers render web pages in x,y co-ordinates (width, height) however they also keep the z-index. The z-index is somewhat controlled by css code. The performance of the page rendering is also controlled by how deep the content is i.e. depth of the displayable html element. Performance degrades as we move the elements deep inside the container elements. e.g.

<div><div><div><h3>hello world!</h3></div></div></div>

Some javascript operations (which operates on DOM structure internally, e.g. getelementbyid), also get affected by how deep the target element lies as it needs to traverse through the DOM tree.

Figuring out the depth of the element in the static html is easy however when it comes to browser, it all depends on how we generate final html code, using javascript,css as Dynamic Html. In this case the final view of the html codebase can be viewed only by the browser's internal memory.

I tried to play with the browser ('Firefox') tool and found very interesting web page architectures (because they look like amazing colorful physical designs) of google and HN home pages.

feel free to explore the gallery below:

[gallery link="file"]

Getting started with Cassandra on Windows

Running Cassandra on Windows: step by step presentation

...follow below steps to successfully setup and run sample test case...

1. Download Cassandra from http://cassandra.apache.org/download/​

2. unzip under C:\cassandra\apache-cassandra-0.7.8

3. set CASSANDRA_HOME=C:\cassandra\apache-cassandra-0.7.8 and make sure JAVA_HOME​ is already set.

4. modify config/cassandra.yaml

  and change all the directories to windows. e.g.
--------------------------------------------------------------------------------

data_file_directories:

- C:/tools/apache-cassandra-0.7.8/data/data
commitlog_directory: C:/tools/apache-cassandra-0.7.8/data/commitlog

saved_caches_directory: C:/tools/apache-cassandra-0.7.8/data/saved_caches

--------------------------------------------------------------------------------


now create below folders under C:/tools/apache-cassandra-0.7.8/data/:

 

callouts, bootstrap, staging, saved_caches, commitlog, data

-these folders are used for diferent purposes.

5. run cassandra.bat  - it runs cassandra server

6. run cassandra-cli.bat -it runs client console

execute below in client console:
__________________________________________

get sample user details from config/password.properties

e.g.   jsmith=havebadpass

connect using connect 127.0.0.1/9160

or

connect 127.0.0.1/9160  jsmith 'havebadpass';

or

c:\cassandra\apache-cassandra-0.7.8\bin>cassandra-cli.bat  --host localhost

----------------------------------------------------------------------------------------

 

Starting Cassandra Client

Welcome to cassandra CLI.

Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit.

[default@unknown] 

===========Connect to Cassandra server===============

[default@unknown] connect 127.0.0.1/9160  jsmith 'havebadpass';

Connected to: "Test Cluster" on 127.0.0.1/9160

[jsmith@unknown] connect 127.0.0.1/9160;

Connected to: "Test Cluster" on 127.0.0.1/9160

===========Create KeySpace===============

[jsmith@unknown] create keyspace Twissandra;

b97c69f2-bc7c-11e0-84c7-e700f669bcfc

Waiting for schema agreement...

... schemas agree across the cluster

===========Set KeySpace===============

[jsmith@unknown] use Twissandra;

Authenticated to keyspace: Twissandra

===========Create Column Family 'user'===============

[default@Twissandra] create column family User with comparator = UTF8Type;

c273cb23-bc7c-11e0-84c7-e700f669bcfc

Waiting for schema agreement...

... schemas agree across the cluster

===========Insert Sample User 'jsmith'===============

[default@Twissandra] set User['jsmith']['first'] = 'John';

Value inserted.

===========Fetch Sample User 'jsmith'===============

[default@Twissandra]  get User['jsmith'];

=> (column=first, value=4a6f686e, timestamp=1312230501056000)

Returned 1 results.

[default@Twissandra]

----------------------------------------------------------------------------------------​

__________________________________________

 

More details on executing sample queries can be found at:http://wiki.apache.org/cassandra/CassandraCli

Or you could refer my previous post @ useful-bookmarks-on-cassandra

 

Why Java will be dominating language in 2013

Here are the few developments which might outshine other languages:

1. Java SE 8

Java 8 is expected in September 2013 and will include at a minimum the features that were planned for Java 7 but later deferred.

• Language-level support for lambda expressions (officially, lambda expressions; unofficially, closures) under Project Lambda.There was an ongoing debate in the Java community on whether to add support for lambda expressions. Sun later declared that lambda expressions would be included in Java and asked for community input to refine the feature.


• Parts of project Coin that are not included in Java 7


• JSR 310: Date and Time API


• Tight integration with JavaFX

2. Powerful JVM languages

Year 2012 was the start of JVM languages where Scala picked up pretty well. There are positive development on Akka, Play 2.0, Groovy, Closure, JRuby and Kotlin. It all depends on the developers and how the consolidation and adoption of these languages will look like in 2013.

3. Integration of JavaScript in Java language

OpenJDK will be integrating JavaScript in Java SDK and will make it part of Java 8. There are chances that Node.js will get entry into this development.

4. Improved performance with GPU

OpenJDK has initiative/project (Sumatra) which tries to improve the performance by utilizing GPU.

5. Java in the cloud

With the advent of frameworks like IaaS and PaaS, there is possibility of introducing jcloud in Java SE8. There might be more 3rd party cloud providers in supporting the java in the cloud.

 

 

Useful bookmarks on Cassandra

Cassandra Links...

• Locking and Txn: http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/​

• DataStax: http://www.datastax.com/docs/0.8/index​


• Visual Data Model: ​http://www.javageneration.com/?p=70


• Data Model: http://wiki.apache.org/cassandra/DataModel


• Intro: http://nosql.mypopescu.com/post/573604395/tutorial-getting-started-with-cassandra


• Presentation: http://nosql.mypopescu.com/post/517791994/presentation-gary-dusbabek-rackspace-on-cassandra


• http://nosql.mypopescu.com/post/385372130/your-chance-to-review-the-fosdem-nosql-event#fosdem-ericevans​


• Writing: http://www.mikeperham.com/2010/03/13/cassandra-internals-writing/


• Up& Running:http://blog.evanweaver.com/2009/07/06/up-and-running-with-cassandra/


• Write Path: http://wiki.apache.org/cassandra/ArchitectureInternals#Write_path


• Read Path: http://wiki.apache.org/cassandra/ArchitectureInternals#Read_path


• Deletes & Failure Detection: http://wiki.apache.org/cassandra/ArchitectureInternals#Deletes


• Slides & Videos of Cassandra Summit: http://www.datastax.com/dev/blog/slides-and-videos-cassandra-summit-2010


• Operations: http://wiki.apache.org/cassandra/Operations


• Data & Setup: http://javamaster.wordpress.com/2010/03/22/apache-cassandra-quick-tour/


• Multi Node cluser: http://wiki.apache.org/cassandra/MultinodeCluster


• Twissandra source code: https://github.com/twissandra/twissandra


• Hector examples: https://github.com/zznate/hector-examples​


• Hector: https://github.com/rantav/hector


• Data Model in Java: http://schabby.de/cassandra-getting-started/


• Cassandra GUI: http://code.google.com/p/cassandra-gui/


• Hector: http://www.datastax.com/sites/default/files/hector-v2-client-doc.pdf​

How to take screenshot on Mac Book Air / Pro laptops

Here are the 3 different ways (by Anson Alexandar) to take screenshot on Mac laptops.

1. Whole Screen, press "Command + Shift + 3": it takes screenshot of whole screen

2. A region on the screen, press "Command + Shift + 4": changes mouse pointer which helps to select a specific region of the screen and takes snapshot when mouse click is released

3. A window, press "Command + Shift + 4 + Space Bar": changes mouse pointer to camera icon. now click on any window and it takes snapshot of that window

 

Note that these screenshots will get saved on desktop.

getting started with GIT - an introduction

Git is an open source distributed version control system.

The screen shots were taken from youtube presentation by Scott Chacon.

Traditional version control systems used to have a single storage where all the developers used to connect and operate. This lead to a single point of failure.

In git, there is no single server which maintains the history. All parties/developers's machine become node in the distributed system and they can work in isolated fashion as well. Basically the system behaves as loosely coupled.

Below is the comparison between traditional VCS and Git. Git stores the file contents along with the checksum which relieves it from managing the change log and plenty of indirected connections between file versions.

Git stores all the checksum as directed graph which helps it easily identify what and how to merge code.

Below are the basic commands for doing any file modifications:

when we do 'git checkout' it creates index database and then creates a local directory where stores all the contents. Running 'git add' puts the file contents along with the checksum in the Index database. The final 'git commit' does a commit from Index database to local repository. Note that 'commit' doesn't look at the local files, it only looks at Index database for final commit to repository. What it means is after doing 'add', we can delete the local file however 'commit' will still continue without loosing the change.

Merging: git does merging by looking at the changes between current branch 'master' and the branch to be merged 'branchA'. It basically walks through the directed graph and tries to find the common ancestor. If found then it does a fast-forward merge which means it moves the head pointer to 'branchA'.

Simple merge (fast forward). merge of branch iss53:

Simple delete: if Head can reach the branch directly then it deletes else skips. so it has a safety mechanism.

git branch -d iss53

i18n can't be reached from head so using below command forcefully deletes it. Hard delete.

git branch -D i18n

steps of code push.

scott pushes the change

git does a fast-forward merge in origin master repo because it sees a common ancestor between origin master and scott's master.

Now Nick tries to push but that will fail as origin master in Nic's local repo and in origin remote repo are pointing to different commit versions.

so Nick does a 'git fetch' first.

then he does merge. because (pull = fetch + merge).

now things look good and origin master points to right commit version. and hence the collaborated delivery completed gracefully,

How to create Aliases?

Git subsets

answer is: git log i18n ^master

What is the change in iss53 which is not in i18n?

answer is the highlighted commit histories:

...Quizes...

Git Incoming? its the change which is in origin/master and not in local master. solution: fetch will pull in the change.

here is how to check what changes we are missing in local:

git log origin/master ^master

Git outgoing? its the change which is in local master and still not got pushed to origin/master. solution: git push will deliver the change

here is how to check what changes we still did not merge to origin/master from local:

git log master ^origin/master

Which is bigger e^pi or pi^e? - a mathematical approach

Here are the mathematical definitions of e and π (pi):


π = Pi = 22/7  (22/7 is approximate value of pi more accurate value is 355/113)

Now we will take mathematical approach in solving the equality problem.

Lets Assume pi^e > e^pi

now taking natural log (Log of base e) on both sides.

Log (pi^e) > Log (e^pi)

eLog(pi) > piLog(e)

eLog(pi) > pi          ---> Log e is 1 because its Log of base e. (LOGe(e) = 1)

Log(pi) > pi/e

since we know that Pi > 1 and hence Log(pi) >1,

then above becomes
1 > pi/e
e > pi        - which is false  (pi = 3.14, e = 2.72)

Therefore our assumption (pi^e > e^pi) is not correct

hence pi^e  is less than e^pi.

Add Application to Windows context menu

Adding an Application to the Right Click on Every Folder

Here is how to add any application to the Context Menu when you right click on any Folder. This way you do not have to always go to the Start Menu. When you right click on any folder, you can have access to that application, the same as using Sent To.

1. Open RegEdit
2. Go to HKEY_CLASSES_ROOT\Folder\shell
3. Add a new Key to the "Shell" Key and name it anything you like.
4. Give it a default value that will appear when you right click a folder, i.e. NewKey (use an "&" without the quotes, in front of any character and it will allow you to use the keyboard)
5. Click on the Key HKEY_CLASSES_ROOT\Folder\shell\NewKey
6. Add a New Key named Command
7. Set the (Default) value of the application you want to run
8. For example: c:\program files\internet explorer\iexplore.exe (Include the full path and parameters if you need them)



 

How a computer scientist failed in solving Fermat Theorem?

Fermat's last theorem was "states that no three positive integers a, b, and c can satisfy the equation a^n + b^n = c^n for any integer value of n greater than two".



here is the puzzle:

A computer scientist claims that he proved Fermat theorem is correct for the following 3 numbers:

x=2233445566, y=7788990011, z=9988776655

He declared that the below equation satisfies for N:
x^N + y^N = z^N

Later the scientist was proved wrong by a 10 year old kid. How come he do that even without the help of the computers?

Here is the solution:

x=2233445566, y=7788990011, z=9988776655

square the last digits of each numbers and notice that the last digit of the result remains same.
which proves that for any value of N, the last digits remains same.

6 * 6 = 36 , 1 * 1 = 1 , 5 * 5 = 25

hence x^N ends with 6,  y^N ends with 1,  z^N ends with 5

try out the equality with the last digits:  6 + 1 != 5

this proves that the values of x,y and z doesn't satisfy Fermat's theorem.