A collection of "low level" JVM and JavaScript related articles and more

Here's a nice collection of "low level" JVM and JavaScript related articles. None of which would be on anyone's list for low level programming.

While doing some reading on math and matrix operations in Java I came across many projects trying to overcome the limitations of the JVM while trying to implement numerical recipes:

• A good (slightly old) paper on ways to store matrices more efficiently
• Some evaluation of the different libraries that do efficient, element wise matrix operations
• More interestingly, we still don't have SSE support in Java for matrix operations - a request that has been open since 2007

While we are stuck with this, Dart seems to be making better progress by supporting SIMD instructions. JavaScript is getting weirder ("low level") with Emscripten and Asm.js.

Other interesting Java related reading material:

• Unsafe and lazy set - I've long wondered what lazySet() did
• Packed Objects - tuples in the JVM
• An evaluation of collections in Java for primitives
• Transform your JARs into executables
• JVM heap dumps using GDB
• I had no idea "top -H" would display all threads in a process/JVM
• Crash the JVM instantly to see if your app can survive crashes (hint: Unsafe.putLong(0, 0) to force a segfault)
• Disruptor + log4j2 = very fast, async logging
• If you've never seen Doug Lea on stage before - Future of the JVM, panel discussion
• Charlie Hunt – The Fundamentals of JVM Tuning, a nice tutorial
• Gallery of processor cache effects 
• Two good Cassandra tech talks

More next time! (Of course)

Graphs, machine learning, PostGres and other tidbits

I hadn't pushed out my "favorite reads of the season" for a while. So, here's a bunch of links to keep you occupied over the next few days.

Graphs, search and recommendations:

• Under the Hood: Building out the infrastructure for Graph Search
• LinkedIn's Cleo and Search (Cleo claims to have been inspired by FB's graph search)
• GraphLab - another take on large scale, distributed graph processing. Similar to Apache Giraph but not based on Hadoop code.
• Graph Based Recommendation Systems at eBay. Graphs, algebra and Cassandra. (I had to go back to basics to understand slide 9)

Statistics, machine learning presentations and resources:

• Data, Data, Data: Thousands of Public Data Sources
• Scalable and Flexible Machine Learning With Scala @ LinkedIn
• Big Data, Small Computers (H2O)
• A Practical Intro to Data Science
• Numeric Programming in Scala with Spire

A couple of peculiar, networking related blog entries.

 

Discussion on Redis mailing list about SSD / Twitter Fatcache / Facebook McDipper and a follow up.

 

While doing some research on NoSQL systems, especially Cassandra, I was surprised to hear that newer releases of Cassandra are moving away from the flexible, semi-structured column families. Instead with CQL, there is a well somewhat restrictive, repetitive schema that should work well for certain workloads. Is it me or does it look like NoSQL is grudgingly moving towards SQL?
 

Speaking of SQL, PostGres is moving in the other direction. Recent (9.x+) versions have some very interesting column data types - Array, HSTORE, JSON etc. Of course, its SQL support is obviously fantastic.

And finally, a nice talk on trade processing and a of paper on MongoDB for finance.


Ashwin.

Importing OpenSSL/EC2 .pem keypair to Java keystore

I spent several hours scouring the internet looking for a way to import (OpenSSL) Amazon EC2's .pem keypair into a Java keystore. At the end of this frustrating exercise I was baffled to see how scattered the information was.

(FYI, doing this on Windows, especially the OpenSSL interaction part, for self-signing a certificate was painful even with Cygwin. I had to resort to using my Linux distro running in a VM)

To save myself time in the future and for those of you tearing your hair out looking for the same information, here it is. (The file paths are not real. You have to clean them up to match your setup):

Here are my references in no particular order:

• http://stackoverflow.com/questions/2813843/possible-reasons-for-timeout-when-trying-to-access-ec2-instance 
• https://forums.aws.amazon.com/message.jspa?messageID=334402 
• https://confluence.atlassian.com/display/JIRA/Connecting+to+SSL+services
• http://www.agentbob.info/agentbob/79-AB.html
• http://www.startux.de/java/44-dealing-with-java-keystores
• http://codeartisan.blogspot.com/2009/05/public-key-cryptography-in-java.html
• http://wiki.eclipse.org/Generating_a_Private_Key_and_a_Keystore
• http://stackoverflow.com/questions/2484026/import-pem-public-and-private-keys-to-jks-keystore
• http://www.akadia.com/services/ssh_test_certificate.html

To complement this, there were other things I had to do (being a first time user of EC2) to make my EC2 instance accept SSH connections:

• http://blog.taggesell.de/?/archives/73-Managing-Amazon-EC2-SSH-login-and-protecting-your-instances.html
• http://dotnetbutchering.blogspot.com/2011/07/how-to-ssh-into-amazon-ec2-linux.html
• http://stackoverflow.com/questions/8193768/trying-to-ssh-into-an-amazon-ec2-instance-permission-error
• https://help.ubuntu.com/community/EC2StartersGuide#Official_Ubuntu_Cloud_Guest_Amazon_Machine_Images_.28AMIs.29
• https://www.openshift.com/page/install-and-setup-putty-ssh-client-for-windows

And then to install Oracle JDK 7 on my EC2 Ububtu image:

• http://thedaneshproject.com/posts/how-to-install-java-7-on-ubuntu-12-04-lts/

Ashwin.

Information Overload

Information overload  a.k.a:

• Too many websites, blogs, apps, social networks and not enough unification (and time)
• Whatever happened to open formats? (ahem.. RSS/Atom?)

YesSQL, JVMs that need to be NUMA aware & other stories

Here's a whole bunch of fascinating reading material I've accumulated these past few months. You can tell there's a lot of love going on for SQL/RDBMS. Then some crazy JVM deployments that make you sit up and wonder. There's also quite a bit of performance related articles on UI/browser technologies. 

Data tier:

• Using Arrays as Materialized Paths in Postgres
• The Etsy Shard Architecture: Starts With S and Ends With Hard
• Under the Hood: Automated backups
• Databus: LinkedIn's low latency change data capture system
• WAL internals of PostgreSQL
• SQL on SSDs: Hot and Crazy Love
• Sharding & IDs at Instagram
• Counting Triangles Faster

 JVM:

• How much memory objects take on JVMs
• NUMA-Aware Java Heaps for in-memory databases (Watch for the line that says "heaps approaching 1TB" in size! Of course they mean using DirectBuffers but still..)

Here's a nice tool that I've filed for later. Esp useful if you find yourself doing production/support calls - Your logs are your data: logstash + elasticsearch. Sort of a poor man's Splunk. 

UI (mostly beating the life out of HTTP and JavaScript):

• Faster Websites: Crash Course on Web Performance
• How browsers work
• WebKit in Your Living Room
• Building the Netflix UI for Wii U
• High Performance Networking in Google Chrome
• Talks To Help You Become A Better Front-End Engineer In 2013 (There are some good ones there)

After covering all 3 tiers - DBs, JVMs and UIs, why stop there when you can finish it off by learning something about QA/unit testing? Here are some relatively new JUnit features (they've finally caught up with TestNG):

JUnit:

• Rules
• Parameterized tests and its better description here
• Assumptions
• Theories and its nicer description here
• Matchers

That should keep you busy for several days. Until next time!