
* (DONE) zookeeper data => /data/db/zookeeper/
*        chown zookeeper:zookeeper /var/zookeeper
*        zookeeper txlog => ??somewhere??
* (DONE) hbase site.xml => fix the hdfs to be a hostname not IP


Mozilla's Socorro project uses HBase as the primary data store for crash reports.  The current production cluster is 17+2 nodes.  We are moving the app to a new datacenter in the next two months and will be spinning up a 30 node cluster there.

We are currently running 0.20.6, but are striving to upgrade to .89 quickly to pick up some important fixes for issues we are hitting.

We have 17k regions.  We store about 2.5M crash reports per day and process 10% of them which are available on the live http://crash-stats.mozilla.com/ site.

Our servers are dual quad cores with 24GB of RAM and 3 1.5TB disks.

17k regions
2.5M docs/day

17 * 8 cores / 24GB / 4.5TB


http://hbase.markmail.org/thread/sp5p7yc73l3kjhuk




---------------------------------------------------------------------------


flying monkey => hbase
* flume?
* rest?

hbase write into table - while read from other tables?
hbase LZO


  Java  --N--> HBase
  Java  --∞--> HBase
  HBase --1--> Ruby
  
  HBase --N--> Pig
  Ruby  --1--> HBase
  Pig   --N--> Hbase


twitter datasets loaded
apeyeye datasets loaded
apeyeye handler
hbase => trstrank

schemas
- size cluster in regions? blocks? NN ram? hb master ram? rs ram?

monitoring
extra cluster roles
lzo?

nn metadata safety
ebs volumes

compute vs production clusters -- how? S3?


