A collection of "low level" JVM and JavaScript related articles and more

Here's a nice collection of "low level" JVM and JavaScript related articles. None of which would be on anyone's list for low level programming.

While doing some reading on math and matrix operations in Java I came across many projects trying to overcome the limitations of the JVM while trying to implement numerical recipes:

• A good (slightly old) paper on ways to store matrices more efficiently
• Some evaluation of the different libraries that do efficient, element wise matrix operations
• More interestingly, we still don't have SSE support in Java for matrix operations - a request that has been open since 2007

While we are stuck with this, Dart seems to be making better progress by supporting SIMD instructions. JavaScript is getting weirder ("low level") with Emscripten and Asm.js.

Other interesting Java related reading material:

• Unsafe and lazy set - I've long wondered what lazySet() did
• Packed Objects - tuples in the JVM
• An evaluation of collections in Java for primitives
• Transform your JARs into executables
• JVM heap dumps using GDB
• I had no idea "top -H" would display all threads in a process/JVM
• Crash the JVM instantly to see if your app can survive crashes (hint: Unsafe.putLong(0, 0) to force a segfault)
• Disruptor + log4j2 = very fast, async logging
• If you've never seen Doug Lea on stage before - Future of the JVM, panel discussion
• Charlie Hunt – The Fundamentals of JVM Tuning, a nice tutorial
• Gallery of processor cache effects 
• Two good Cassandra tech talks

More next time! (Of course)

Graphs, machine learning, PostGres and other tidbits

I hadn't pushed out my "favorite reads of the season" for a while. So, here's a bunch of links to keep you occupied over the next few days.

Graphs, search and recommendations:

• Under the Hood: Building out the infrastructure for Graph Search
• LinkedIn's Cleo and Search (Cleo claims to have been inspired by FB's graph search)
• GraphLab - another take on large scale, distributed graph processing. Similar to Apache Giraph but not based on Hadoop code.
• Graph Based Recommendation Systems at eBay. Graphs, algebra and Cassandra. (I had to go back to basics to understand slide 9)

Statistics, machine learning presentations and resources:

• Data, Data, Data: Thousands of Public Data Sources
• Scalable and Flexible Machine Learning With Scala @ LinkedIn
• Big Data, Small Computers (H2O)
• A Practical Intro to Data Science
• Numeric Programming in Scala with Spire

A couple of peculiar, networking related blog entries.

 

Discussion on Redis mailing list about SSD / Twitter Fatcache / Facebook McDipper and a follow up.

 

While doing some research on NoSQL systems, especially Cassandra, I was surprised to hear that newer releases of Cassandra are moving away from the flexible, semi-structured column families. Instead with CQL, there is a well somewhat restrictive, repetitive schema that should work well for certain workloads. Is it me or does it look like NoSQL is grudgingly moving towards SQL?
 

Speaking of SQL, PostGres is moving in the other direction. Recent (9.x+) versions have some very interesting column data types - Array, HSTORE, JSON etc. Of course, its SQL support is obviously fantastic.

And finally, a nice talk on trade processing and a of paper on MongoDB for finance.


Ashwin.

Importing OpenSSL/EC2 .pem keypair to Java keystore

I spent several hours scouring the internet looking for a way to import (OpenSSL) Amazon EC2's .pem keypair into a Java keystore. At the end of this frustrating exercise I was baffled to see how scattered the information was.

(FYI, doing this on Windows, especially the OpenSSL interaction part, for self-signing a certificate was painful even with Cygwin. I had to resort to using my Linux distro running in a VM)

To save myself time in the future and for those of you tearing your hair out looking for the same information, here it is. (The file paths are not real. You have to clean them up to match your setup):

Here are my references in no particular order:

• http://stackoverflow.com/questions/2813843/possible-reasons-for-timeout-when-trying-to-access-ec2-instance 
• https://forums.aws.amazon.com/message.jspa?messageID=334402 
• https://confluence.atlassian.com/display/JIRA/Connecting+to+SSL+services
• http://www.agentbob.info/agentbob/79-AB.html
• http://www.startux.de/java/44-dealing-with-java-keystores
• http://codeartisan.blogspot.com/2009/05/public-key-cryptography-in-java.html
• http://wiki.eclipse.org/Generating_a_Private_Key_and_a_Keystore
• http://stackoverflow.com/questions/2484026/import-pem-public-and-private-keys-to-jks-keystore
• http://www.akadia.com/services/ssh_test_certificate.html

To complement this, there were other things I had to do (being a first time user of EC2) to make my EC2 instance accept SSH connections:

• http://blog.taggesell.de/?/archives/73-Managing-Amazon-EC2-SSH-login-and-protecting-your-instances.html
• http://dotnetbutchering.blogspot.com/2011/07/how-to-ssh-into-amazon-ec2-linux.html
• http://stackoverflow.com/questions/8193768/trying-to-ssh-into-an-amazon-ec2-instance-permission-error
• https://help.ubuntu.com/community/EC2StartersGuide#Official_Ubuntu_Cloud_Guest_Amazon_Machine_Images_.28AMIs.29
• https://www.openshift.com/page/install-and-setup-putty-ssh-client-for-windows

And then to install Oracle JDK 7 on my EC2 Ububtu image:

• http://thedaneshproject.com/posts/how-to-install-java-7-on-ubuntu-12-04-lts/

Ashwin.

Information Overload

Information overload  a.k.a:

• Too many websites, blogs, apps, social networks and not enough unification (and time)
• Whatever happened to open formats? (ahem.. RSS/Atom?)

YesSQL, JVMs that need to be NUMA aware & other stories

Here's a whole bunch of fascinating reading material I've accumulated these past few months. You can tell there's a lot of love going on for SQL/RDBMS. Then some crazy JVM deployments that make you sit up and wonder. There's also quite a bit of performance related articles on UI/browser technologies. 

Data tier:

• Using Arrays as Materialized Paths in Postgres
• The Etsy Shard Architecture: Starts With S and Ends With Hard
• Under the Hood: Automated backups
• Databus: LinkedIn's low latency change data capture system
• WAL internals of PostgreSQL
• SQL on SSDs: Hot and Crazy Love
• Sharding & IDs at Instagram
• Counting Triangles Faster

 JVM:

• How much memory objects take on JVMs
• NUMA-Aware Java Heaps for in-memory databases (Watch for the line that says "heaps approaching 1TB" in size! Of course they mean using DirectBuffers but still..)

Here's a nice tool that I've filed for later. Esp useful if you find yourself doing production/support calls - Your logs are your data: logstash + elasticsearch. Sort of a poor man's Splunk. 

UI (mostly beating the life out of HTTP and JavaScript):

• Faster Websites: Crash Course on Web Performance
• How browsers work
• WebKit in Your Living Room
• Building the Netflix UI for Wii U
• High Performance Networking in Google Chrome
• Talks To Help You Become A Better Front-End Engineer In 2013 (There are some good ones there)

After covering all 3 tiers - DBs, JVMs and UIs, why stop there when you can finish it off by learning something about QA/unit testing? Here are some relatively new JUnit features (they've finally caught up with TestNG):

JUnit:

• Rules
• Parameterized tests and its better description here
• Assumptions
• Theories and its nicer description here
• Matchers

That should keep you busy for several days. Until next time!

Trip to Disneyand, Anaheim

We did a 5 day trip to Disneyland, Anaheim and Universal Studios over the winter break.

Disneyland during X'mas and New Year's eve is a wonderful, lively place to be at. Having spent 3.5 days at Disneyland and California Adventure park around the 25th, we felt we got our money's worth (and more) from the 4-day park hopper tickets. The festive atmosphere in spite of the manageable crowd was as good as going on a road trip to a National Park.

New Year's even is probably more crowded than X'mas. We did not stay to find out.

One our way back we decided to spend one evening and the next day till noon at Universal Studios, LA. Universal in LA does not have as many real rides as the one in Orlando. What they have here are mostly 3-D, virtual rides. Some of them are pretty good but we still missed the old fashioned, gravity induced roller coaster rides.

Some things that we should've done/Note to self:

• If you are driving down from Bay Area to LA and Pacheco pass/CA-152 is blocked/crowded, just continue down CA-101 and go to I-5 via Coalinga/CA-198. It's a little longer but was desolate when we drove
• Book via AAA and save on parking fees at Disney (offers might keep changing)
• Book via CityPass and save on our combo Disney-Universal entry fees (check if there are black-out dates)
• Book cheaper hotel outside Anaheim since we had a car to drive around. We cancelled and booked at Fullerton after spending a night at an expensive but dingy hotel opposite Disneyland
• All these parks charge a parking fee
• The hotels also charge a parking fee (!) even if you book a room
• Hotels in Fullerton are cheaper (like the Fullerton Marriott and much nicer)
• Hotels a little further away from Universal Studios are also cheaper
• Use a good phone app to book hotels like Kayak or HotelTonight or the latest new app
• Wear good shoes as there always is a lot of walking and standing in lines

Some photos:

(CA-198)

(Disneyland and California Adventure Park)

(Over the Tejon pass)

(Pyramid Lake)

Have a wonderful new year ahead!

Starving CPUs, slow multi-dim arrays in Java, cool infrastructure & more

A short list of articles/papers that I thought were worth sharing:

CPU and Assembly:

• Towards native performance using Java (from the OpenGamma guys)
• Starving CPUs (from the Python PyTable developer)
• UTF-8 processing using SIMD

Polyglot analytics/programming - how startups are going full speed with Redis & MongoDB for simple analytics. Of course this leaves the heavy duty, complex analytics to Hadoop and columnar data warehouses.

• Kicking ass with Redis - for real world problems
• The beauty of simplicity - Mastering database design with Redis
• Redis - The AK-47 of post-relational databases (good content, bad title)
• Crashlytics on Redis
• Powerful autocomplete with Redis
• Mongodb for analytics
• Redis and Jedis - Delightfully simple and focused NoSQL (My own, quick intro)

Really educational, large scale, infrastructure talks:

• Petabyte Scale Data at Facebook
• From Zero to 500k QPS in Three Years: Scaling AppNexus
• Real-Time Delivery Architecture at Twitter (Lot of Redis in there)
• Netflix infrastructure (Lot of Cassandra)

Some misc notes:

• Today being Friday.. here's something for your entertainment (Hint: Linus & C++) 
• A quick note on DNS caching in Java
• Kurt Thomas on Security (Overview of the state of the art in security and cyber crime)

Happy holidays and a happy new year!