Tuesday, May 24, 2011

A few points while choosing Hadoop to solve our problems


Update your Open Source Tools frequently
                Open source technologies are those being contributed from various developers across the globe, there are not much of rigorous testing happening behind the hood before the open source technologies are released compared to other propriety products like Oracle, Informatica etc. When we use Hadoop tools, every tool is getting improved day by day with new patches on to it and new major releases rolled out frequently. It is highly recommended that we keep our work confined to the latest releases of open source tools. In Hadoop we use a variety of tools like Pig, Hive, Hbase, Sqoop etc, There should be an effective mechanism in cluster management that would keep track of the latest updates available on the softwares , and tools , and in turn keep it updated with the new releases. This practice would considerably reduce the wastage of development time in debugging a few issues that has been fixed already in latest releases. Such an incremental approach to cluster management is very essential in cases of technologies like Hadoop which is still not cent percent stable.

Choose your Open Source Tool wisely
                This is a crucial choice for m/r developers. When we are heading to the implementation of a requirement the choice of tools plays a vital role in the long term existence and smooth running of the project. We have to choose only those open source Software/tool that has a big name associated with it or supporting the same ie a pool of developers are constantly working on its improvement.  This is a matter of high priority because in most scenarios an open source tool/project is short lived if not adopted by a major IT giant. Hence choose your tool wisely. If you need an interactive distributed data base what would you go for, HBase? Why Hbase? Is it adopted my any IT major? These queries should be addressed before you finalize your tool .

Minimize Custom Map Reduce Codes
                A small question to you all, in the web era if you want to develop a web scale application, what would be your preferred language of development? Assembly Language or any High Level Language. Definitely it would be a High Level Language. The same applies in the Hadoop scenario as well, always depend on High level tools on Hadoop built over map reduce such as Pig, Hive etc for your application. These tools are already highly optimized for better performance and effective resource utilization. You should go for a custom map reduce if and only if it is inevitable, when you are developing a custom Map Reduce code a lot of effort has to go behind in fine tuning the same for better performance and effective cluster utilization. 
 It is hard for any map reduce developer to take into account the configurable parameters in hadoop which counts to more than 300. 

26 comments:

  1. tutorials on Upgrade Hadoop is excellent.I am happy to found such helpful and fascinating post that is written in well manner. i actually enhanced my data when browse your post .thanks.
    Hadoop Training in hyderabad

    ReplyDelete
  2. Nice piece of article you have shared here, my dream of becoming a hadoop professional become true with the help of Hadoop Course in Chennai, keep up your good work of sharing quality articles.

    big data training in velachery|hadoop training chennai velachery|hadoop training institute in t nagar

    ReplyDelete
  3. Your information is really useful for me.Thanks for sharing such a valuable information. AWS Training in chennai | AWS Training chennai | AWS course in chennai

    ReplyDelete
  4. very nice blogs!!! i have to learning for lot of information for this sites...Sharing for wonderful information.Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing. cloud computing training in chennai | cloud computing training chennai | cloud computing course in chennai | cloud computing course chennai

    ReplyDelete
  5. I gathered a lot of information through this article.Every example is easy to understandable and explaining the logic easily.Thanks! VMWare Training in chennai | VMWare Training chennai | VMWare course in chennai | VMWare course chennai

    ReplyDelete
  6. Oracle DBA Training in Chennai
    Thanks for sharing this informative blog. I did Oracle DBA Certification in Greens Technology at Adyar. This is really useful for me to make a bright career..

    ReplyDelete
  7. Whatever we gathered information from the blogs, we should implement that in practically then only we can understand that exact thing clearly,
    but it’s no need to do it, because you have explained the concepts very well. It was crystal clear, keep sharing..
    Websphere Training in Chennai

    ReplyDelete
  8. Data warehousing Training in Chennai
    I am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly..

    ReplyDelete
  9. Selenium Training in Chennai
    Wonderful blog.. Thanks for sharing informative blog.. its very useful to me..

    ReplyDelete
  10. Oracle Training in chennai
    Thanks for sharing such a great information..Its really nice and informative..

    ReplyDelete

  11. SAP Training in Chennai
    This post is really nice and informative. The explanation given is really comprehensive and informative..

    ReplyDelete
  12. This information is impressive..I am inspired with your post writing style & how continuously you describe this topic. After reading your post,thanks
    for taking the time to discuss this, I feel happy about it and I love learning more about this topic
    Android Training In Chennai In Chennai

    ReplyDelete
  13. Pretty article! I found some useful information in your blog, it was awesome to read,thanks for sharing this great content to my vision, keep sharing..
    Unix Training In Chennai

    ReplyDelete
  14. I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing..
    SalesForce Training in Chennai

    ReplyDelete
  15. There are lots of information about latest technology and how to get trained in them, like Best Hadoop Training In Chennai in Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies Hadoop Training in Chennai By the way you are running a great blog. Thanks for sharing this blogs..

    ReplyDelete
  16. This is really an awesome article. Thank you for sharing this.It is worth reading for everyone. Visit us:
    Oracle Training in Chennai

    ReplyDelete
  17. very nice blogs!!! i have to learning for lot of information for this sites...Sharing for wonderful information.Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing.Oracle DBA Training in Chennai

    ReplyDelete
  18. Wonderful tips, very helpful well explained. Your post is definitely incredible. I will refer this to my friend.SalesForce Training in Chennai

    ReplyDelete
  19. Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing.Nice article i was really impressed by seeing this article, it was very interesting and it is very useful for me.. Android Training in Chennai

    ReplyDelete
  20. Really awesome blog. Your blog is really useful for me. Thanks for sharing this informative blog. Keep update your blog.SAP Training in Chennai

    ReplyDelete
  21. I found some useful information in your blog,it was awesome to read, thanks for sharing this great content to my vision, keep sharing..selenium Training in Chennai

    ReplyDelete
  22. Excellent information with unique content and it is very useful to know about the information based on blogs. Hadoop Training in Chennai

    ReplyDelete
  23. Hadoop solves the hard scaling problems caused by large amounts of complex data. we are guide to u.please visit us @ Hadoop Training in Chennai |
    Salesforce Training in Chennai |
    Android Training in Chennai |
    Software Testing Training in Chennai

    ReplyDelete
  24. Besant Technologies Offers best Android Training in Chennai with best placement support. The training sessions are handled by working professionals who are experts in teaching Android according to needs of IT companies.
    Android Training in Chennai | Android Training in Chennai |
    Android Training in Chennai |
    Android Training in Chennai |

    ReplyDelete