Suppose we have a Spark RDD, and we want to do transformation on each record by getting the value from an external resource/API/service. How could we do it efficiently?
The most naive way to achieve it can be this:
keyedRDD.map { case(key, record)...Flink provides us the BucketingSink to sink data to HDFS. This article analyzes the BucketingSink...
Recently, we wrote a lot of raw and complex SQL queries for data analysis in our new project, and these queries run on a MySQL database. Then, we notice that it becomes a nightmare to maintain these queries, and we decide to add unit tests to...
It is common that users need to key in their addresses on e-commerce or public service applications. Providing address autocompletion functionality could greatly improve user experience and reduce typo errors. Here, I will provide an address...
It is a common concern for an ecommerce website that customer might paid twice for a certain order, probably due to user's unawareness, front end bugs or browser unresponsiveness during the transaction, which hurts user experience. Thus, backend...
Recently, we set up a new Rails project (named as app below), and our database.yml is shown below, so you can see that it reads from environment variables to set database names and credentials accordingly:
default: &default adapter: postgresql...This article explores the capabilities to alter Java standard library using AspectJ without recompiling source code of Java. It is highly experimental and do not apply it to production environment.
AspectJ...
Due to the dynamic nature of Ruby, it is hard to locate a method definition statically. Thus, we can only find the method definition accurately at runtime. Here are some tips.
Suppose we have an object called foo. We can use foo.methods to find...
Infinite scroll is very popular in web right now because it largely boosts user experience. Thus, I would like to share some tips on implementing a perfect infinite scroll.
Infinite scroll is user friendly, but may not be search engine friendly....
As a Rails developer, you should have worked with Sprocket to deal with front-end resources. This article highlights a key difference between require and require_tree.
When require is used, Sprocket searches the file from search paths. You can...