Java concurrency (multi-threading) - Tutorial

This article describes how to do concurrent programming with Java. It covers the concepts of parallel programming, immutability, threads, the executor framework (thread pools), futures, callables and the fork-join framework.

1. Concurrency

1.1. What is concurrency?


Concurrency is the ability to run several programs or several parts of a program in parallel. If a time consuming task can be performed asynchronously or in parallel, this improve the throughput and the interactivity of the program.
A modern computer has several CPU's or several cores within one CPU. The ability to leverage these multi-cores can be the key for a successful high-volume application.

1.2. Process vs. threads

process runs independently and isolated of other processes. It cannot directly access shared data in other processes. The resources of the process, e.g. memory and CPU time, are allocated to it via the operating system.
thread is a so called lightweight process. It has its own call stack, but can access shared data of other threads in the same process. Every thread has its own memory cache. If a thread reads shared data it stores this data in its own memory cache. A thread can re-read the shared data.
A Java application runs by default in one process. Within a Java application you work with several threads to achieve parallel processing or asynchronous behavior.

2. Improvements and issues with concurrency

Within a Java application you work with several threads to achieve parallel processing or asynchronous behavior.

2.1. Limits of concurrency gains

Concurrency promises to perform certain task faster as these tasks can be divided into subtasks and these subtasks can be executed in parallel. Of course the runtime is limited by parts of the task which can be performed in parallel.
The theoretical possible performance gain can be calculated by the following rule which is referred to as Amdahl's Law.
If F is the percentage of the program which can not run in parallel and N is the number of processes, then the maximum performance gain is 1/ (F+ ((1-F)/n)).

2.2. Concurrency issues

Threads have their own call stack, but can also access shared data. Therefore you have two basic problems, visibility and access problems.
A visibility problem occurs if thread A reads shared data which is later changed by thread B and thread A is unaware of this change.
An access problem can occur if several thread access and change the same shared data at the same time.
Visibility and access problem can lead to
  • Liveness failure: The program does not react anymore due to problems in the concurrent access of data, e.g. deadlocks.
  • Safety failure: The program creates incorrect data.

3. Concurrency in Java

3.1. Processes and Threads

A Java program runs in its own process and by default in one thread. Java supports threads as part of the Java language via the Thread code. The Java application can create new threads via this class.
Java 1.5 also provides improved support for concurrency with the in the java.util.concurrent package.

3.2. Locks and thread synchronization

Java provides locks to protect certain parts of the code to be executed by several threads at the same time. The simplest way of locking a certain method or Java class is to define the method or class with the synchronizedkeyword.
The synchronized keyword in Java ensures:
  • that only a single thread can execute a block of code at the same time
  • that each thread entering a synchronized block of code sees the effects of all previous modifications that were guarded by the same lock
Synchronization is necessary for mutually exclusive access to blocks of and for reliable communication between threads.
You can use the synchronized keyword for the definition of a method. This would ensure that only one thread can enter this method at the same time. Another threads which is calling this method would wait until the first threads leaves this method.
public synchronized void critial() {
  // some thread critical stuff
  // here
} 
You can also use the synchronized keyword to protect blocks of code within a method. This block is guarded by a key, which can be either a string or an object. This key is called the lock. All code which is protected by the same lock can only be executed by one thread at the same time
For example the following datastructure will ensure that only one thread can access the inner block of the add() andnext() methods.
package de.vogella.pagerank.crawler;

import java.util.ArrayList;
import java.util.List;

/** * Data structure for a web crawler. Keeps track of the visited sites and keeps * a list of sites which needs still to be crawled. * * @author Lars Vogel * */
public class CrawledSites { private List crawledSites = new ArrayList(); private List linkedSites = new ArrayList(); public void add(String site) { synchronized (this) { if (!crawledSites.contains(site)) { linkedSites.add(site); } } }
/** * Get next site to crawl. Can return null (if nothing to crawl) */
public String next() { if (linkedSites.size() == 0) { return null; } synchronized (this) { // Need to check again if size has changed if (linkedSites.size() > 0) { String s = linkedSites.get(0); linkedSites.remove(0); crawledSites.add(s); return s; } return null; } } }

3.3. Volatile

If a variable is declared with the volatile keyword then it is guaranteed that any thread that reads the field will see the most recently written value. The volatile keyword will not perform any mutual exclusive lock on the variable.
As of Java 5 write access to a volatile variable will also update non-volatile variables which were modified by the same thread. This can also be used to update values within a reference variable, e.g. for a volatile variable person. In this case you must use a temporary variable person and use the setter to initialize the variable and then assign the temporary variable to the final variable. This will then make the address changes of this variable and the values visible to other threads.

4. The Java memory model

4.1. Overview

The Java memory model describes the communication between the memory of the threads and the main memory of the application.
It defines the rules how changes in the memory done by threads are propagated to other threads. The Java memory model also defines the situations in which a thread re-fresh its own memory from the main memory.
It also describes which operations are atomic and the ordering of the operations.

4.2. Atomic operation

An atomic operation is an operation which is performed as a single unit of work without the possibility of interference from other operations.
The Java language specification guarantees that reading or writing a variable is an atomic operation(unless the variable is of type long or double). Operations variables of type long or double are only atomic if they declared with the volatile keyword. .
Assume i is defined as int. The i++ (increment) operation it not an atomic operation in Java. This also applies for the other numeric types, e.g. long. etc).
The i++ operation first reads the value which is currently stored in i (atomic operations) and then it adds one to it (atomic operation). But between the read and the write the value of i might have changed.
Since Java 1.5 the java language provides atomic variables, e.g. AtomicInteger or AtomicLong which provide methods like getAndDecrement()getAndIncrement() and getAndSet() which are atomic.

4.3. Memory updates in synchronized code

The Java memory model guarantees that each thread entering a synchronized block of code sees the effects of all previous modifications that were guarded by the same lock.

5. Immutability and Defensive Copies

5.1. Immutability

The simplest way to avoid problems with concurrency is to share only immutable data between threads. Immutable data is data which cannot changed.
To make a class immutable make
  • all its fields final
  • the class declared as final
  • the this reference is not allowed to escape during construction
  • Any fields which refer to mutable data objects are
    • private
    • have no setter method
    • they are never directly returned of otherwise exposed to a caller
    • if they are changed internally in the class this change is not visible and has no effect outside of the class
An immutable class may have some mutable data which is uses to manages its state but from the outside this class nor any attribute of this class can get changed.
For all mutable fields, e.g. Arrays, that are passed from the outside to the class during the construction phase, the class needs to make a defensive-copy of the elements to make sure that no other object from the outside still can change the data

5.2. Defensive Copies

You must protected your classes from calling code. Assume that calling code will do its best to change your data in a way you didn't expect it. While this is especially true in case of immutable data it is also true for non-immutable data which you still not expect that this data is changed outside your class.
To protect your class against that you should copy data you receive and only return copies of data to calling code.
The following example creates a copy of a list (ArrayList) and returns only the copy of the list. This way the client of this class cannot remove elements from the list.
package de.vogella.performance.defensivecopy;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;

public class MyDataStructure {
  List list = new ArrayList();

  public void add(String s) {
    list.add(s);
  }

  
/** * Makes a defensive copy of the List and return it * This way cannot modify the list itself * * @return List */
public List getList() { return Collections.unmodifiableList(list); } }

6. Threads in Java

The base means for concurrency are is the java.lang.Threads class. A Thread executes an object of typejava.lang.Runnable.
Runnable is an interface with defines the run() method. This method is called by the Thread object and contains the work which should be done. Therefore the "Runnable" is the task to perform. The Thread is the worker who is doing this task.
The following demonstrates a task (Runnable) which counts the sum of a given range of numbers. Create a new Java project called de.vogella.concurrency.threads for the example code of this section.
package de.vogella.concurrency.threads;

/** * MyRunnable will count the sum of the number from 1 to the parameter * countUntil and then write the result to the console. * * MyRunnable is the task which will be performed * * @author Lars Vogel * */
public class MyRunnable implements Runnable { private final long countUntil; MyRunnable(long countUntil) { this.countUntil = countUntil; } @Override public void run() { long sum = 0; for (long i = 1; i < countUntil; i++) { sum += i; } System.out.println(sum); } }
The following example demonstrate the usage of the Thread and the Runnable class.
package de.vogella.concurrency.threads;

import java.util.ArrayList;
import java.util.List;

public class Main {

  public static void main(String[] args) {
    // We will store the threads so that we can check if they are done
    List threads = new ArrayList();
    // We will create 500 threads
    for (int i = 0; i < 500; i++) {
      Runnable task = new MyRunnable(10000000L + i);
      Thread worker = new Thread(task);
      // We can set the name of the thread
      worker.setName(String.valueOf(i));
      // Start the thread, never call method run() direct
      worker.start();
      // Remember the thread for later usage
      threads.add(worker);
    }
    int running = 0;
    do {
      running = 0;
      for (Thread thread : threads) {
        if (thread.isAlive()) {
          running++;
        }
      }
      System.out.println("We have " + running + " running threads. ");
    } while (running > 0);

  }
} 
Using the Thread class directly has the following disadvantages.
  • Creating a new thread causes some performance overhead
  • Too many threads can lead to reduced performance, as the CPU needs to switch between these threads.
  • You cannot easily control the number of threads, therefore you may run into out of memory errors due to too many threads.
The java.util.concurrent package offers improved support for concurrency compared to the direct usage ofThreads. This package is described in the next section.

7. Threads pools with the Executor Framework

Tip

You find this examples in the source section in Java project calledde.vogella.concurrency.threadpools.
Thread pools manage a pool of worker threads. The thread pools contains a work queue which holds tasks waiting to get executed.
A thread pool can be described as a collection of Runnable objects (work queue) and a connections of running threads. These threads are constantly running and are checking the work query for new work. If there is new work to be done they execute this Runnable. The Thread class itself provides a method, e.g. execute(Runnable r) to add a new Runnable object to the work queue.
The Executor framework provides example implementation of the java.util.concurrent.Executor interface, e.g. Executors.newFixedThreadPool(int n) which will create n worker threads. The ExecutorService adds life cycle methods to the Executor, which allows to shutdown the Executor and to wait for termination.

Tip

If you want to use one thread pool with one thread which executes several runnables you can use the Executors.newSingleThreadExecutor() method.
Create again the Runnable.
package de.vogella.concurrency.threadpools;

/** * MyRunnable will count the sum of the number from 1 to the parameter * countUntil and then write the result to the console. * * MyRunnable is the task which will be performed * * @author Lars Vogel * */
public class MyRunnable implements Runnable { private final long countUntil; MyRunnable(long countUntil) { this.countUntil = countUntil; } @Override public void run() { long sum = 0; for (long i = 1; i < countUntil; i++) { sum += i; } System.out.println(sum); } }
Now you run your runnables with the executor framework.
package de.vogella.concurrency.threadpools;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Main {
  private static final int NTHREDS = 10;

  public static void main(String[] args) {
    ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
    for (int i = 0; i < 500; i++) {
      Runnable worker = new MyRunnable(10000000L + i);
      executor.execute(worker);
    }
    // This will make the executor accept no new threads
    // and finish all existing threads in the queue
    executor.shutdown();
    // Wait until all threads are finish
    executor.awaitTermination();
    System.out.println("Finished all threads");
  }
} 
In case the threads should return some value (result-bearing threads) then you can use thejava.util.concurrent.Callable class.

8. Futures and Callables

The code examples for this section are created in a Java project called de.vogella.concurrency.callables.
The executor framework presented in the last chapter works with Runnables. Runnable do not return result.
In case you expect your threads to return a computed result you can use java.util.concurrent.Callable. TheCallable object allows to return values after completion.
The Callable object uses generics to define the type of object which is returned.
If you submit a Callable object to an Executor the framework returns an object of typejava.util.concurrent.Future. This Future object can be used to check the status of a Callable and to retrieve the result from the Callable.
On the Executor you can use the method submit to submit a Callable and to get a future. To retrieve the result of the future use the get() method.
package de.vogella.concurrency.callables;

import java.util.concurrent.Callable;

public class MyCallable implements Callable {
  @Override
  public Long call() throws Exception {
    long sum = 0;
    for (long i = 0; i <= 100; i++) {
      sum += i;
    }
    return sum;
  }

} 
package de.vogella.concurrency.callables;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class CallableFutures {
  private static final int NTHREDS = 10;

  public static void main(String[] args) {

    ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
    List> list = new ArrayList>();
    for (int i = 0; i < 20000; i++) {
      Callable worker = new MyCallable();
      Future submit = executor.submit(worker);
      list.add(submit);
    }
    long sum = 0;
    System.out.println(list.size());
    // now retrieve the result
    for (Future future : list) {
      try {
        sum += future.get();
      } catch (InterruptedException e) {
        e.printStackTrace();
      } catch (ExecutionException e) {
        e.printStackTrace();
      }
    }
    System.out.println(sum);
    executor.shutdown();
  }
} 

9. Nonblocking algorithms

Java 5.0 provides supports for additional atomic operations. This allows to develop algorithm which are non-blocking algorithm, e.g. which do not require synchronization, but are based on low-level atomic hardware primitives such as compare-and-swap (CAS). A compare-and-swap operation check if the variable has a certain value and if it has this value it will perform this operation.
Non-blocking algorithm are usually much faster then blocking algorithms as the synchronization of threads appears on a much finer level (hardware).
For example this created a non-blocking counter which always increases. This example is contained in the project called de.vogella.concurrency.nonblocking.counter.
package de.vogella.concurrency.nonblocking.counter;

import java.util.concurrent.atomic.AtomicInteger;

public class Counter {
  private AtomicInteger value = new AtomicInteger(); 
  public int getValue(){
    return value.get();
  }
  public int increment(){
    return value.incrementAndGet();
  }
  
  // Alternative implementation as increment but just make the 
  // implementation explicit
  public int incrementLongVersion(){
    int oldValue = value.get();
    while (!value.compareAndSet(oldValue, oldValue+1)){
       oldValue = value.get();
    }
    return oldValue+1;
  }
  
} 
And a test.
package de.vogella.concurrency.nonblocking.counter;

import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class Test {
    private static final int NTHREDS = 10;

    public static void main(String[] args) {
      final Counter counter = new Counter();
      List> list = new ArrayList>();

      ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
      for (int i = 0; i < 500; i++) {
        Callable worker = new  Callable() {
          @Override
          public Integer call() throws Exception {
            int number = counter.increment();
            System.out.println(number);
            return number ;
          }
        };
        Future submit= executor.submit(worker);
        list.add(submit);

      }
      
      
      // This will make the executor accept no new threads
      // and finish all existing threads in the queue
      executor.shutdown();
      // Wait until all threads are finish
      while (!executor.isTerminated()) {
      }
      Set set = new HashSet();
      for (Future future : list) {
        try {
          set.add(future.get());
        } catch (InterruptedException e) {
          e.printStackTrace();
        } catch (ExecutionException e) {
          e.printStackTrace();
        }
      }
      if (list.size()!=set.size()){
        throw new RuntimeException("Double-entries!!!"); 
      }

    }


} 
The interesting part is how the incrementAndGet() method is implemented. It uses a CAS operation.
public final int incrementAndGet() {
        for (;;) {
            int current = get();
            int next = current + 1;
            if (compareAndSet(current, next))
                return next;
        }
    } 
The JDK itself makes more and more use of non-blocking algorithms to increase performance for every developer. Developing correct non-blocking algorithm is not a trivial task.
For more information on non-blocking algorithm, e.g. examples for a non-blocking Stack and non-block LinkedList, please see http://www.ibm.com/developerworks/java/library/j-jtp04186/index.html

10. Fork-Join in Java 7

Java 7 introduce a new parallel mechanism for compute intensive tasks, the fork-join framework. The fork-join framework allows you to distribute a certain task on several workers and then wait for the result.
E For Java 6.0 you can download the package (jsr166y) from Download site
For testing create the Java project "de.vogella.performance.forkjoin". If you are not using Java 7 you also need tojsr166y.jar to the classpath.
Create first a algorithm package and then the following class.
package algorithm;

import java.util.Random;

/** * * This class defines a long list of integers which defines the problem we will * later try to solve * */
public class Problem { private final int[] list = new int[2000000]; public Problem() { Random generator = new Random(19580427); for (int i = 0; i < list.length; i++) { list[i] = generator.nextInt(500000); } } public int[] getList() { return list; } }
Define now the Solver class as shown in the following example coding.

Tip

The API defines other top classes, e.g. RecursiveAction, AsyncAction. Check the Javadoc for details.
package algorithm;

import java.util.Arrays;

import jsr166y.forkjoin.RecursiveAction;

public class Solver extends RecursiveAction {
  private int[] list;
  public long result;

  public Solver(int[] array) {
    this.list = array;
  }

  @Override
  protected void compute() {
    if (list.length == 1) {
      result = list[0];
    } else {
      int midpoint = list.length / 2;
      int[] l1 = Arrays.copyOfRange(list, 0, midpoint);
      int[] l2 = Arrays.copyOfRange(list, midpoint, list.length);
      Solver s1 = new Solver(l1);
      Solver s2 = new Solver(l2);
      forkJoin(s1, s2);
      result = s1.result + s2.result;
    }
  }
} 
Now define a small test class for testing it efficiency.
package testing;

import jsr166y.forkjoin.ForkJoinExecutor;
import jsr166y.forkjoin.ForkJoinPool;
import algorithm.Problem;
import algorithm.Solver;

public class Test {

  public static void main(String[] args) {
    Problem test = new Problem();
    // check the number of available processors
    int nThreads = Runtime.getRuntime().availableProcessors();
    System.out.println(nThreads);
    Solver mfj = new Solver(test.getList());
    ForkJoinExecutor pool = new ForkJoinPool(nThreads);
    pool.invoke(mfj);
    long result = mfj.getResult();
    System.out.println("Done. Result: " + result);
    long sum = 0;
    // check if the result was ok
    for (int i = 0; i < test.getList().length; i++) {
      sum += test.getList()[i];
    }
    System.out.println("Done. Result: " + sum);
  }
} 

11. Deadlock

A concurrent application has the risk of a deadlock. A set of processes are deadlocked if all processes are waiting for an event which another process in the same set has to cause.
For example if thread A waits for a lock on object Z which thread B holds and thread B wait for a look on object Y which is hold be process A then these two processes are looked and cannot continue in their processing.

How to find out if your computer has a virus ?

If you use the Internet, your computer is at risk of infection from viruses.  Much like biological viruses, some are harmless, some are merely annoying and some can make your life hell. Even if you only occasionally use your home computer, it is important that you understand the risks and know how to protect yourself.

What is a virus?

In the 1990s the public's impression of what a virus is came from the media. Newspapers reported how  programs such as Michaelangelo[1]  would could cause a financial appocalypse by infecting computers around the world and deleting vital data. If a virus announces its presence in such a melodramatic way as shutting down your PC or deleting your data, although you might find it infuriating, at least you know that you have a problem. Far more dangerous are the new breed of crimeware viruses, unwelcome programs that run unnoticed while you continue to use your computer, blissfully unaware that you are at risk.
 
A virus is a form of malware (malicious software)Malware is software on your computer that you didn't intentionally install, that is running against your wishes and against your interests. Precisely defining the distinction between a virus and other types of malware can get confusing and the definitions are constantly evolving. For now just rememeber that a virus is a form of malware and that malware is bad news for you and your computer.
 

Why do people make viruses and other malware?

Some eccentric people make viruses for fun or as a personal challenge, but these are generally harmless or annoying at worst. A lot of malware is now written with truly criminal intent and is designed to acheive financial gain for their creator at your expense. Don't assume that the target is only big business because home users are equally at risk. Here are some common motives for creating malware.

Stealing your bank details

Your bank will have warned you that shopping on the Internet involves a risk of people stealing your financial details. This can result either from inadequate security considerations by the online seller or because you have some form of malware on your computer.  This is a scary thought when you consider that certain transactions can take money from your account immediately if the criminal has access to the right details.

Stealing your identity

Even if you don't shop online, you could potentially end up in a lot of difficulty if someone steals your identity. Many websites request that you provide them with personal details that may seem innocuous enough, but to criminals can provide the first step to identity theft. A gradual accumulation of your personal details collected by malware could be sufficient to enable someone to build up a profile of your identity and, for example, apply for credit in your name. You might not find out about this until the credit company has started hassling you for the repayments.

Using your computer for illegal activity

Have you ever wondered where all those spam e-mails come from offering investment tips, cheap viagra and breast enlargemnts?  One way that spammers get junk mail delivered is to send it using software, known as bots,  running secretly on other people's computers.  With millions of computers connected to the Internet, malware that uses innocent home users' equipment for dubious or illegal activities can put a lot of power in the hands of deviant people.

Find out if you're infected

By now you should understand that malware is a bad thing and it can affect you.  One last misconception needs to be addressed:
 
Misconception: 'You only get viruses from dodgy looking websites and emails'.
 
The reason why this isn't true will be explained later. For now, let's address the problem.
 

Antivirus software

The practical defence for home users is antivirus software. Crucially this software must be up-to-date. If your computer came with trial antivirus software that has now expired it's as good as useless. This is because there are hundreds of new threats crawling the Internet every year and if your antivirus software hasn't received its updates, it can't defend you against an attack. So, if you don't have antivirus software or if your software is out of date, you must install an antivirus program.  Here's how:

1.  Manually air gap the computer

Disconnect your computer from the Internet immediately, preferably physically (i.e. disconnect your modem or network cable).
 

2. Download antivirus software from a trusted source using a clean computer.

Use a computer that is already running up-to-date reputable antivirus software to download the latest antivirus installation software, then burn the installation files CD or DVD. If you don't want to pay for antivirus software, some antivirus companies offer free antivirus software for home use. Make sure you download from a source that has a reputation for virus free downloads such as www.download.com. Avast! Home Edition and AVG Antivirus Free Edtion are two popular antivirus products that are free for home use, easy to download and easy to install[2].
 
If downloading and burning isn't possible or seems too complicated you're probably going to have to buy some software from your local computer shop. There are many commercial antivirus products available off-the-shelf  for home use. Ask the dealer which product will suit you best. 

3. Stop suspicious software from running

Kill any processes that are running that you suspect may be malware.  To do this, bring up Task Manager (by pressing Control, Alt and Delete at the same time and then clicking on Task Manager) then look through the list of processes on the processes tab.  Make a note of the names of all the processes, then using an uninfected computer, look up the names of these processes on a reliable site If possible, manually stop any unidentified or malware processes on the infected machine using the End Process button from Task Manager.  You should also try to stop any unnecessary or suspicious applications from running using the End Task button on the Applications tab in Task Manager.  Leave your computer running in this condition as malware is likely to restart if you restart the infected computer.
 

4. Backup your data

Backup your important personal files to CD, DVD or another form of removable media. Beware that these files may contain infected material so put a warning label on the disk.  These backups are for an emergency restore only, such as if the files on your hard drive are wiped unintentionally later on in the procedure or if the antivirus installation prevents the computer from starting correctly.

5. Install antivirus sof tware and scan for viruses.

It is preferable to install the antivirus software in a diagnostic mode such as safe mode in Microsoft Windows[3].  Malware and extraneous operating system services are less likely to be running on you computer if you are in safe mode, hence it is less likely that the installation can be sabotaged or conflict with another program. Not all antivirus programs will allow you to perform an install in safe mode.  In that case, at least make sure that suspicious and unneccesary processes are not running by following the instructions in Step 3 before installing the antivirus software.
 
Follow the instructions supplied by your antivirus software providor to complete the installation.  This will usually involve restarting the computer and automatically retrieving the latest antivirus updates from the Internet.
 
Once you have completed the installation you should use the software to run a scan of your computer.  Depending on the age and type of your setup this may take hours, so have patience.  Once the scan is complete your antivirus software should present you with some reassuring information - either that the machine is clean or that malware has been detected that can now be bannished.  The exact procedure will vary depending on your antivirus software.
 
Make another backup of the now (hopefully) clean personal data files to CD or DVD.  If you are intending to or have to do a low-level reinstallation (see Advanced Techniques), use these backups to restore your personal data rather than those you made in step 1. 

6. Get advice on identity theft.

If you suspect that you may have been the victim of malware you need to prepare for the possibility that your identity has been stolen.  Advice may vary depending on your country, but a good starting point will  be contacting your bank.  You may need to renew accounts and cards or even file a police report.  In all cases you must keep a close watch on any future bank statements. 
If you are still having problems then you probably fall into one of two categories - either your operating system is not applicable to the process presented above or you are dealing with a particularly nasty virus that requires a more complex removal process.  In either case, if you're not confident or able to deal with the attack using advanced techniques (see 'Advanced malware removal') then you will probably have to get help from a trusted person more technical than yourself to help you.  If in doubt, contact your Internet Service Provider for advice.
 

Isn't there a faster detection method?

There are some common symptoms that arouse immediate suspicion including:
  • Computers mysteriously shutting down on their own. 
  • Programs running excessively slowly.
  • Unfamiliar processes running on the computer.
  • Unfamilar programs starting on their own or duplicating themselves.
  • Other unexpected computer behaviour.
 
Unfortunately, the most insidious attacks will not be apparent to the victim until it's too late, so looking out for syptoms alone is not a substitute for installing antivirus software. Ultimately there is no guarantee that your computer is not infected with malware. New attacks are becoming ever more sophisticated and many attacks will innevitable infect home systems before the antivirus updates are available, however, antivirus software is currently the most reliable and practical way for a home user to reduce the risk of a problem and to detect infection.
 

How do you get a virus?

Returning to the question of how malware is transmitted in the first place, it should be repeated that in the current age this is not limited to getting infected through contact with dubious material such as Internet porn sites, pirated games and unsolicited e-mails.  Although these traditional sources still propogate malware, many of today's attacks employs more subtle and sophisticated methods than, for example, the Anna Kournikova virus[4].

The Blaster worm

Blaster was a malicious program that spread itself over the Internet to Windows XP and Windows 2000 computers in 2003.  One of the syptoms was quite dramatic, effectively making a computer unusable by forcing it to shut down within seconds of booting up. Most of the high profile viruses in the recent years up until then had spread through email attachments and required a bit of assistance from the user themselves, but Blaster could spread over a network without the user being involved.  Computers without the latest Windows updates or the protection of a firewall were vulnerable merely by being connected to the Internet.

Phishing sites

Your bank has probably warned you about phishing (pronounced 'fishing') sites. Users are directed to the phishing site from a phishing email - a bogus but official looking electronic communication pursuading you to visit the phishing site. By presenting a web page that looks identical and could even appear to have the same URL as a familar trustworthy site, such as your online bank, the phishing site lures you into following instructions or submitting information in the belief that you are safe, when in fact you are submitting information to a criminal or assisting them with the installation of malware on your computer. 
 
A variation on this idea is that gaining the victim's trust by appearing that you are there to help them with an urgent problem. Some malicious webpages disguise themselves as a warning messages claiming that you have a virus but that it can be removed by following certain instructions.  Those instructions then acheive the exact opposite - exposing the computer to an attack and installing the malware.
 

Auto run from CDs, DVDs and pendrives

The PC's autorun or autoplay feature was once very useful as it enabled you to insert a disk, such as a data CD, and the software on the disk would start automatically.  As long as you only inserted media that came from a reputable source, you could be pretty sure that this feature wasn't going to automatically run any malware because in the old days, CDs read only for most users, so malware was not able to write itself to the disk media in the first place.
 
Today most users have CD or DVD burners and can use pen drives (also known as 'USB sticks' or 'memory sticks').  Although lots of CDs and DVDs are still read only, pen drives are almost always the opposite.  With the autorun feature still enabled on most computers, malware can easily install itself to and from pen drives that are inserted into your computer.  You can stop autorun from functioning by holding down the shift key while you insert the pen drive or CD. To disable the feature more permanently is a bit more complicated, but instructions are provided in a Microsoft knowledge base article.
 
The unprotected transfer of data with pen drives is so prolific at the moment, it is suspected malware has even managed to make its way onto the International Space Station using this method [5].

Staying protected

The battle against malicious attacks is an exercise in risk management and as always there is a trade off between risk and cost.  For example, when Blaster was spreading itselft over the Internet, the protected Windows XP/2000 users were those with either firewall software (which was less common at the time) or both the latest antivirus updates and the latest operating system updates.  Should you therefore religiously download and install the latest operating system updates?  Not necessarily.  Even the more rigorously tested and less frequently released service packs may not be suitable to install on your computer[6], although this is the exception rather than the rule.
 
So what can you do?  Whilst there's guaranteed solution to the problem, there are steps you can take that are not too complicated, time consuming or expensive and will help to keep you protected:
 
  1. Always have antivirus software installed and up to date. Check for web browser and OS updates regularly.
  2. Use 'strong' passwords and don't reveal them to anyone.
  3. Don't use an administrator account if you don't have to.  Most modern operating systems support accounts with different levels of privelege.  If you're just surfing the web, you don't need to be logged on as the system administrator and your normal user account doesn't need administrative priviledges.
  4. Verify the authenticity of websites and emails that request information.  Check that the webpages and emails come from the company they claim to be and are not a clever typographical variation on the company name. Be suspicious of any email requesting personal or financial information and ignore all spam mail.
  5. Don't download or install software from an untrusted source and hold down the shift key when inserting pen drives or other types of media.
  6. Use an e-mail service that scans emails for malware and don't open email attachments from an untrusted source, even if apparently forwarded by friends.
  7. Use a personal firewall. Most commercial home operating systems now come with a free firewall built in. If not, use a third party personal firewall product designed for your operating system.
  8. If while browsing the Internet you start to receive messages claiming that you have a virus, exit your web browser, disconnect from the Internet and restart your computer.  Once restarted, if you genuinely have a malware problem, your antivirus software will inform you after downloading the latest updates and doing a scan.
  9. Be suspicious of instuctions from unverifiable sources.  If an someone tells you to manually adjust the configuration of your computer, find out what the risks are and try to understand what your browser configurations actually do[7].  Never assume that an unverified source is an innocent source because it has plausible motives. A plausible positive motive is exactly what malicious attackers use as their disguise.  
In general you should be vigilant regarding computer security and not just where it concerns malware. If personal or financial data needs to be kept secret then it must be sent using a secure web page. This is usually signified by an address starting with 'https' instead of 'http' and a symbol presented by the web browser (not the webpage) such as a padlock.  Find out exactly what this should look like in your web browser so you don't fall for fakes and remember that this is only protecting the information in transit, you still need to be sure that the recipient is trustworthy.  Also, be aware that a standard e-mail is not a secure private communication - the data is very easy to read in transit by someone snooping with the right equipment. If you do send payment details over the Internet use a credit card.  Credit cards are less risky than debit cards as the seller does not receive immediate payment, so you may have time to cancel the transaction if you realise that you're being ripped off.

Advanced malware removal

Re-installation

This may seem like overkill, but re-installation is the most reliable way to get rid of a virus. The lower the level at which you can do this the better, because the aim of some malware is to entrench itself at as low a level as possible. So if you know how, format your hard drive, then re-install the OS with current anti-virus before restoring your applications and data.  Obviously you are in a better position to do this if you are pre-prepared.  Sometimes when you buy your computer it will come with a restoration disk that can be used to return the machine to its factory condition.  Keep this safe.  If not, find out from the manufacturers/suppliers if your computer has an alternative factory restoration method, and if not, look in to preparing your own restoration procedure.

Malware removal tools

Specific malware often needs a specific removal tool in addition to the standard antivirus software. These can often be obtained for free from antivirus companies.  Semantec, for example, provides a list of recent removal tools.
Remember: the main threat to your computer is no longer a tennager writing writing programs for kicks in his bedroom - it's organised crime. There is a growing blackmarket in malware for criminal purposes[8], and the new demand is for malware that goes undetected, so be as thorough as you can in cleaning your system.
 

Recent improvements from Microsoft

When Blaster attacked, a typical Microsoft user had no firewall, no antispyware and was using Windows XP.  From a security point of view, Microsoft's latest offerings are an improvement, but the attacks are also becoming more sophisticated.  Windows Defender (anti-spyware) and Windows Firewall are free security products that now come as standard with the Windows operating system.  Windows 7 and Windows Vista have a feature called User Account Control (UAC) that is designed to protect the user by alerting them whenever they attempt to use administrator privileges regardless of the authority of the account logged in.

Microsoft have recently entered into the free antivirus mark.  Microsoft Security Essentials is intended to be a comprehensive anti-malware package free to users who have a genuine copy of Windows. Reviews of the product have been generally positive in comparison to the other free alternatives[9].
 

Some jargon explained

The information presented here has been aimed at people who don't know whether they have an infected computer or not and are therefore more interested in the consequences of malware rather than the technical vocabulary. Some of the more common terms used in relation to home computer security are explained below to help readers with further research.
 
Worms -  Worms are a special category of virus. The most significant distinction between a normal virus and a worm is that worms travel from one computer to another without the user being involved, whereas normal viruses are introduced unintentionally by the user.
 
Trojan Horse - as the name implies, a trojan horse may not appear to be anything dangerous, but it secretly provides an enemy with a way into somewhere they shouldn't be.  A typical trojan horse program will create a 'back door' to your computer, enabling attackers to get at your private data or control your computer remotely.
 
Adware - this is software that routinely delivers unsolicited adverts to the user. Sometimes the inclusion of adware in ligitimate software packages subsidizes the cost of providing the useful software to home users, i.e. you get some free software but you have to put up with the adverts.  This is why adware is not generally considered malware, even if it is annoying, and anti-adware may not be included for free with your antivirus product.
 
Spyware - spyware is a form of malware whose purpose is to monitor your activity and report it to someone else without you knowing about it. Some spyware products are not necessarily destructive but are often objectionable and an invasion of privacy, other products are designed for just plain theft of private information.  The level of spying could be anything from monitoring which websites you go to on the Internet to keystroke logging, which is the electronic equivalent of having someone constantly looking over your shoulder as you type.
 
Firewall - think of a firewall as a monitored gateway between your computer and the network it's connected to. Any communication between your computer and the outside world is subject to the firewall allowing that communication through.  Most firewalls are not intelligent enough to understand if the communication is malicious or not, instead they restrict communication with your computer to approved applications (such as your web browser), or designated channels of communcation (known as ports).  If communication is attempted over a disallowed port or is started by a disallowed application the firewall will block the commication.
 
Phishing and Pharming - Phishing is described above.  Pharming is a very similar concept, but instead of using e-mails as the lure, pharmers use alternative ways to direct you to the spoof web page, such as by exploiting an existing vulnerability to redirect you to a spoof web page when you attempt to access your online banking website.

Top 5 Best Databases

As part of the contest we conducted recently, I got many comments from the TechProceed.com readers who choose their favorite database.

Based on this data, the top spot goes to.. MySQL


If you are new to any of the top 5 database mentioned here, please read the rest of the article to understand more about them.





1. MySQL




MySQL is used in almost all the open source web projects that require a database in the back-end. MySQL is part of the powerful LAMP stack along with Linux, Apache and PHP.

This was originally created by a company called MySQL AB, which was acquired by Sun Microsystems, which was acquired by Oracle. Since we don’t what Oracle will do with MySQL database, open source community has created several forks of MySQL including Drizzle and MariaDB.






Following are few key features:
  • Written in C and C++.
  • MyISAM storage uses b-tree disk tables with index compression for high performance.
  • Support for partitioning and replication.
  • Support for Xpath, full text search.
  • Support for stored procedures, triggers, views etc.,
Additional Information:

2. PostgreSQL





PotgreSQL is a open source object-relational database system. It runs on most *nix flavours, Windows and Mac OS. This has full support for joins, views, triggers, stored procedures etc.,

Following are few key features:
  • MVCC – Multi-Version Concurrency Control
  • Hot backups and point-in-time recovery
  • Support for tablespaces
  • Asynchronous replication
  • Highly scalable
Additional Information:

3. Oracle





Oracle is the best database for any mission critical commercial application. Oracle has following four different editions of the database: 1) Enterprise Edition 2) Standard Edition 3) Standard Edition One 4) Express Edition

Following are few key features of the oracle database.
  • Real Application Cluster (RAC)
  • Data Guard for standby database
  • Virtual Private Database
  • Automatic Memory, Storage and Undo Management
  • OLAP, Partitioning, Data Mining
  • Advance Queuing, XML DB, Support for Spatial data
  • Flashback Database, Query, Table and Transaction
Additional Information:

4. SQLite




SQLite does not work like a traditional client-server model with standalone process. Instead, it is a self-contained, server-less SQL database engine.
Main Features of SQLite:
  • Zero configuration with no setup or admin tasks.
  • Complete database is stored in a single disk file.
  • No external dependencies
  • Supports database of several TB in size
  • Work on most *nix flavors, Mac OS X, windows. It’s also cross-platform.
  • WinCE is supported out-of-the box
Additional Information:

5. Microsoft SQL Server

This is Microsoft’s flagship Database product. If you are stuck in a company that heavily uses Microsoft products, you might end-up working on MS SQL Server.