Amazon S3. Easy, reliable and fast cloud storage


PHP OBSERVER

Amazon S3 is good, reliable, secure, fast cloud storage solution to store and share data especially who are dealing with huge data. Sounds interesting!!! Yes and the process is much more interesting. Here I am going to explain how it works and how we can use it effectively.

The concept is “bucket”. “Bucket”???  Yes first we have to create a bucket with unique name to understand, let’s assume it’s a directory. The flexible part of it is we can create subfolder in it and put our contents inside subfolder, though Amazon handles subfolders in its own way not in a conventional folder structure way of popular operating system.

View original post 276 more words

Advertisements

Speech Recognition in Web Form


PHP OBSERVER

Now a days we use speech recognition in various handhold devices like smart phones, tablets etc. Like saying “play album” will start play your album, “open calculator” will open calculator etc.

Lets come to web forms. How about there are a search field and you are searching for contacts by saying “contacts” or filling up name field by saying “John”.
This is very helpful for inputting long sentences and difficult words.

View original post 115 more words

S3FS – Mount Amazon S3 Cloud Storage into Linux System


PHP OBSERVER

As continuation of my previous writing Amazon S3. Easy, Reliable and Fast Cloud Storage, today in this blog we will try to mount Amazon s3 bucket on Ubuntu system. We already have seen there are several advantages of using Amazon s3 storage. Now question is how we can use it in our program.

There are mainly two way to integrate it with our program:-

– Web services provided by Amazon to store and retrieve files.
– Mount s3 bucket into local system.

View original post 600 more words

Background processes ( How dangerous it could be)


As a software developer we face some situation when user response time is very high. For example sometime we need to send 10,000 mails at a time or converting 1000 files to some other formats at a time.

In such cases users need to wait a long time to get response. To avoid this we usually run the task as a background process and show user a message like “Your request is being processed”.

In my case also there was similar situation and I was handling in this way-

for ($i=0;$i<10000;$i++){
  exec("php convertFileAndMail.php $i >/dev/null 2>&1 & echo $!",, &$output);
}

And the server went into inoperable state and and once even my database server which was running in same system crashed.

Finding the reason:

To find the reason I created 2 files

1) loop.php
2) sleep.php

Codes of sleep.php: (sleep for 5 sec and log 2nd argument passed)

<?php
  sleep(5);
  file_put_contents('log.txt',$argv[1]."\n", FILE_APPEND);
?>

Codes of loop.php: (loop and execute sleep.php)

<?php
  $c=100000;
  for($i=0;$i<$c;$i++) {
     exec("php sleep.php $i >/dev/null 2>&1 & echo $!",, &$output);
  }
?>

And then in terminal if you run following command –

[php loop.php]

Within a few seconds this will hang a normal system. 😦

Reason:

If we look into the files there are almost 0 processing power needed to execute the sleep.php because the file only sleeps for 5 sec and write a number to a file.

Then what is wrong?

The file loop.php adding tasks as background processes and sleep.php takes 5 sec to complete due to sleep. Within this 5 sec loop.php adding millions of background processes.

So in the certain time number of simultaneous processes is huge and it takes lots of memory. Which may even cause system breakdown.

Way out and suggestions:

From above example it is very clear that we have to keep number of simultaneous processes less.

There are a few ways to reach it according to your need-

  • Avoid creating background processes.
  • But sometime we need background processes as mentioned above, in that case try to create less number of background processes. May be in one background process you can do all you need.
  • There are some scenario where we will be needing multiple background processes. Here we have few tools like GEARMAN (http://gearman.org) to handle simultaneous background processes. (I will come up with Gearman concept in another blog later).

Hope this will help you to avoid a pothole in your way. 🙂