Tag Archives: Google Cloud Storage

Creating a serverless ETL nirvana using Google BigQuery

Posted on March 30, 2016 by Shine Technologies

Quite a while back, Google released two new features in BigQuery. One was federated sources. A federated source allows you to query external sources, like files in Google Cloud Storage (GCS), directly using SQL. They also gave us user defined functions (UDF) in that … Continue reading →

Posted in databases, Javascript | Tagged , , , , , , , , , , , , | 4 Comments

Google Cloud Dataproc and the 17 minute train challenge

Posted on October 14, 2015 by Shine Technologies

My work commute My commute to and from work on the train is on average 17 minutes. It’s the usual uneventful affair, where the majority of people pass the time by surfing their mobile devices, catching a few Zs, or by reading a … Continue reading →

Posted in DevOps, Linux, Opinion, Tools | Tagged , , , , , , , , , , , , , , , , , , | 2 Comments

Test Driving Google Cloud Dataflow

Posted on February 25, 2015 by Shine Technologies

Back in June 2014, at the annual Google IO in San Francisco, Google unveiled their newest, and much hyped cloud product, Cloud Dataflow. The demo they did that day, using a live twitter feed to analyze supporter sentiment during the … Continue reading →

Posted in Testing | Tagged , , , , , , , , , , , , | Leave a comment