Karmasphere

Enterprise recognition for partners

martinh — Wed, 20 Jan 2010 22:52:05 +0000

It doesn’t surprise us that three of InfoWorld’s 2010 Technology of the Year Awards involved MapReduce, Hadoop and NoSQL technologies. We want to congratulate our partners at Amazon Web Services for extending “to just about every dimension of cloud computing currently known.” And congratulations to the Apache Foundation for its Apache Hadoop, an open source solution for managing very large, parallel processed datasets. As InfoWorld puts it, “Though the concept is simple, the result is powerful.”

Karmasphere Studio HUG Slides

martinh — Mon, 09 Nov 2009 18:22:11 +0000

Our lead engineer, Shevek, gave a 30 minute presentation at the October 21st Bay Area HUG meeting on Yahoo!’s campus. It’s a pretty good introduction to the studio and its ability to help you prototype, deploy and debug Hadoop jobs and to monitor and view the job, cluster and associated file systems on a regular Hadoop distribution or Amazon’s Elastic MapReduce.

Karmasphere Studio for Hadoop

View more documents from Hadoop User Group.

Hadoop Survey

martinh — Fri, 06 Nov 2009 19:44:43 +0000

We came across the stealth mode guys at MapR Technologies recently. They describe what they’re doing as “engineering game-changing Map/Reduce related technologies”. They’re conducting a survey with some interesting questions for Hadoop users. Taking the survey buys you coffee for a few days.

Beta 0.3 Released – Get Your Counters Here!

martinh — Mon, 02 Nov 2009 16:59:03 +0000

Thanks to lots of use and feedback over the past few weeks, we have made further improvements to the Studio tool. In addition to various bug fixes and performance improvements, there are two new pieces of functionality to draw attention to:

Job and Task Counters
File System Status

As usual, we’re interested in feedback either on the mailing list or in private email. For new users, this is available as the current installable version. For existing users, you should get it automatically, depending on your update check interval.

Hadoop Job and Task Counters

We’ve had a lot of requests for the display and visualization of task and job counters. So new to this release is the ability to view and graphically chart counters in Hadoop jobs. You can also save and re-use custom chart templates, which enables you to build your own general, job and/or task-specific counter dashboards.

File System Status

We’ve also added graphical viewing of cluster file systems so you can see their status at a glance.

Demo and Presentation at Bay Area HUG

martinh — Mon, 26 Oct 2009 21:22:39 +0000

The Bay Area Hadoop User Group is a thriving community that we mingled with last week. Thanks to Dekel for the invitation and squeezing Shevek’s presentation in at the last minute. We know we really ought to get a screencast done but, in the meantime, here are the slides that Shevek presented.

Beta Response

martinh — Wed, 14 Oct 2009 14:19:29 +0000

We’ve been bowled over by the response to the announcement of Karmasphere Studio’s product beta at Hadoop World. A ton of people have downloaded and started using it.

The most common question we’ve received is “why NetBeans?”. To our knowledge, it hasn’t been a roadblock to anyone using it but the simple answer is “it was a convenient vehicle to launch our first product”.

More features and functionality are just around the corner. If you haven’t already, please give us feedback. New stuff is based on what we’re being asked for.

Karmasphere Studio Tutorials

dave — Wed, 14 Oct 2009 14:07:53 +0000

We have published a set of tutorials to help users get started and oriented with Karmasphere Studio’s capabilities.

Job Development and Debugging on the Desktop with the Job Developer

Job Deployment, Monitoring and Debugging on a Hadoop Cluster with the Hadoop Manager

Job Deployment, Monitoring and Debugging on Amazon Elastic MapReduce and S3 with the Hadoop Manager

Karmasphere Studio Now in Beta

martinh — Fri, 02 Oct 2009 05:09:25 +0000

Today we’re publishing the first beta version of Karmasphere Studio for Hadoop. It’s available for free as a plugin to NetBeans.

From your Mac, Linux or Windows desktop, you can:

Graphically develop and debug MapReduce applications for Apache Hadoop.
Deploy, monitor and debug your jobs in real-time
Work with clusters running every major version of Hadoop (0.18, 0.19 and 0.20)
Deploy, debug and monitor your jobs and manage clusters in the cloud using Amazon Elastic MapReduce.
Use HDFS and Amazon S3 file systems.
Monitor jobs on clusters and browse HDFS file systems via SSH tunnel or SOCKS proxy.

In the Job Developer, you prototype your MapReduce job without initially needing a cluster, since Hadoop emulation is included. Your job is visualized as a workflow, which can show the results of individual phases when you provide a local sample data set. Edit your code and see the results immediately.

The Hadoop Manager enables you to work with jobs, clusters and file systems. From here you can deploy jobs to one or more clusters, use local, HDFS and Amazon S3 file systems, see a graphical real-time profile of your cluster capacity and job usage, and dive deeper into the status of your job while it’s running.

Cluster integration with Amazon Elastic MapReduce provides insight into all your jobs and temporary clusters, including the ability to browse the cluster’s HDFS and access job logs.

The product also includes through-the-firewall support so that you can monitor jobs on clusters and browse HDFS file systems via SSH tunnel or SOCKS proxy.

We hope you’ll try it and let us know what you like, what you’d like to see changed or added and, since it’s a beta, tell us any problems you have.

Amazon Elastic MapReduce Integration Available

martinh — Fri, 02 Oct 2009 05:09:10 +0000

For those wanting to run jobs in the cloud without setting up their own Hadoop distribution, Amazon Elastic MapReduce gives you the opportunity to spin up a cluster, small or large, for a job when you need it. And to pay for only as much as you use.

This seemed like a natural fit with Karmasphere Studio. So, in the Beta product, we’ve incorporated easy deployment to Elastic MapReduce clusters, support for the Amazon S3 file system and deep insight into your clusters and jobs running on Amazon Elastic MapReduce.

See the status and progress of all your jobs,.
Profile your cluster capacity and job usage graphically.
Examine job steps and logs in real time.
Browse the cluster’s HDFS file system.
Access files on Amazon S3

You can use the Amazon Elastic MapReduce functionality with jobs developed inside or outside Karmasphere Studio, You can use it as a desktop console for jobs you’ve deployed on Amazon Elastic MapReduce from anywhere, including Karmasphere Studio, the Elastic MapReduce console or command line tools.

See us at Hadoop World

Karmasphere — Mon, 28 Sep 2009 04:04:56 +0000

We’ll be at Hadoop World with more news about Karmasphere Studio for Hadoop.

We’re especially interested in hearing about all the enterprise uses of Hadoop popping up and the needs of Hadoop job developers as we evolve the Studio product.

If you want to learn more from us face-to-face at the event, send us an email and we’ll try to coordinate a meeting.