-
Recent Posts
Top Posts & Pages
- About
- Contact
- A Deep Dive into DynamoDB Partitions
- Filling the BigQuery paddling pool from the Kinesis Hosepipe
- The most important thing when picking HTTP status codes
- Orchestrating Tasks Using AWS SWF
- The Emergence of The 3 Towers: DevSecOps
- Shine hosts a successful Digital Leaders Breakfast
- Creating a serverless ETL nirvana using Google BigQuery
- Pablo rocking the stage at Google's annual cloud event!
Tags
- 5G
- A/B testing
- accessibility
- accessors
- Adobe
- Adobe Experience Manager
- AEM
- Agile
- Amazon
- amazon cognito
- angular
- angularjs
- appstore
- Architecture
- aria
- asynchronous
- Asynchronous Programming
- Awards
- aws
- Babel
- Backbone
- backbone.js
- backbonejs
- battlehack
- bem
- Best Practices
- Big Data
- BigQuery
- bluetooth
- BQ
- braintree
- Broadband
- Chart.js
- CI
- citcon-anz-2010
- Cloud
- Cloud Dataflow
- Cluster Computing
- Clustering
- CMS
- co
- code
- codedojo
- Commenting
- community
- comparison
- concordion
- Conference
- continuous delivery
- CoreLocation
- couchdb
- couchtato
- cq5
- css
- cucumber
- customer experience
- CX
- Database
- Databases
- delegate
- Design
- devops
- DevOpsSec
- DI
- dirty checking
- Distruptive
- Documentation
- DynamoDB
- dzone
- ec2
- emberjs
- empathy
- Enterprise
- es6
- ES2015
- ETL
- fiber
- firebase
- fitbit
- fullcalendar
- GCP
- generators
- gigabit
- Git
- Gitflow
- Google BigQuery
- Google Cloud
- google cloud dataflow
- Google Cloud Datastore
- Google Cloud Platform
- Google Cloud Storage
- Google Developer Expert
- Google Maps
- Google Pub/Sub
- GPS
- grunt-clientlibify
- gruntjs
- hackathon
- hacking
- Hadoop
- hardware
- html5
- HTTP
- hudson
- human-centered design research
- IaaS
- identity map
- idm
- in-app
- Innovation Session
- Internet
- Internet of Things
- iOS
- iPhone
- itunes
- jackson
- Java
- JavaOne
- javascript
- JavaServer Pages
- jenkins
- jhipster
- jmeter
- jquery
- jsconfau
- jslint
- json
- JSP
- key value observer
- kids coding
- kinesis
- koa
- lambdas
- lessons
- Location Data
- Loon
- MapReduce
- Mass Insertion
- melbjs
- meteor
- MFA
- microservices
- Mobile
- Model–view–controller
- monitoring
- Multi -Factor-Authentication
- netbiscuits
- NewRelic
- News
- ngaria
- node
- node.js
- node.js
- NoSQL
- notification
- oam
- Objective-C
- OpenCms
- Open source
- oracle access manager
- osdc2103
- pair-programming
- participatory design
- partitions
- patterns
- payments
- paypal
- Performance
- PHP
- phpunit
- Pipelining
- Play
- Play Framework
- preferred vendors
- Programming
- Projections
- Promises
- Provisioned IOPS
- purchasing
- Python
- qualitative
- R
- Rails
- rds
- re: Invent
- React
- ReactiveCocoa
- React Native
- Redis
- Redshift
- release management
- Replication
- REST
- RESTful
- Risk Management
- Ruby
- s3
- Scala
- SDK
- Security
- Sensis
- service design
- Silex
- Simple Workflow
- Simulate
- single page app
- single page apps
- sinon.js
- site monitoring
- Software Development
- Software Engineering
- spa
- Spark
- Speed
- spring
- Spring Data
- Spring Data REST
- Spring Framework
- Spring Security
- SQL
- Streaming
- streams
- styleguide
- SWF
- swift
- swipeconference
- tech
- Telstra
- testflightapp
- testing
- TLS/SSL
- Tomcat
- Touch ID
- tutorial
- UDF
- UI
- Unit Testing
- usability
- user defined functions
- ux
- wds2013
- web
- web directions
- Web Directions Code
- XCode
- YARN
- YouTube
- YOW!
Meta
Tag Archives: Cloud
Creating a serverless ETL nirvana using Google BigQuery
Quite a while back, Google released two new features in BigQuery. One was federated sources. A federated source allows you to query external sources, like files in Google Cloud Storage (GCS), directly using SQL. They also gave us user defined functions (UDF) in that … Continue reading
Posted in databases, Javascript
Tagged Big Data, Cloud, Database, Google BigQuery, Google Cloud, Google Cloud Platform, Google Cloud Storage, IaaS, javascript, Programming, SQL, UDF, user defined functions
4 Comments
Pablo rocking the stage at Google’s annual cloud event!
Last week, Shine’s very own Pablo Caif gave a presentation at GCP Next 2016 in San Francisco, which is Google’s largest annual cloud platform event. Pablo delivered an outstanding talk on the work Shine have done for Telstra, which involves building solutions on the GCP … Continue reading
Messages in the sky
One of the projects that I’m currently working on is developing a solution whereby millions of rows per hour are streamed real-time into Google BigQuery. This data is then available for immediate analysis by the business. The business likes this. … Continue reading
Google Cloud Dataproc and the 17 minute train challenge
My work commute My commute to and from work on the train is on average 17 minutes. It’s the usual uneventful affair, where the majority of people pass the time by surfing their mobile devices, catching a few Zs, or by reading a … Continue reading
Posted in DevOps, Linux, Opinion, Tools
Tagged Big Data, Cloud, Cluster Computing, Clustering, Google BigQuery, Google Cloud, google cloud dataflow, Google Cloud Platform, Google Cloud Storage, Hadoop, IaaS, Java, MapReduce, Python, R, Scala, Software Engineering, Spark, YARN
2 Comments
Upcoming Shine Event: Leveraging Big Data in the Cloud using Google and AWS
Shine would like to invite you to a special event we are hosting in Melbourne on July 8. In a one-off presentation to be held at ACMI in Federation Square, Shiners Graham Polley and Shane Neubauer will share with you … Continue reading
Tagged Amazon, aws, Big Data, Cloud, GCP, Google, Google Cloud, News
Leave a comment
Playing with Play Framework 2.3.x: REST, pipelines, and Scala
It’s an established trend in the modern software world that if you want to get something done, you’ll probably need to put together a web service to get do it. People expect data and services to be available everywhere, in a mobile world. With the plethora of frameworks and technologies available to go about implementing a web service, it becomes a chore to try using anything beyond what’s already familiar. But every now and then it’s an enjoyable experience to dive into something new and distinctly unfamiliar. Continue reading
Test Driving Google Cloud Dataflow
Back in June 2014, at the annual Google IO in San Francisco, Google unveiled their newest, and much hyped cloud product, Cloud Dataflow. The demo they did that day, using a live twitter feed to analyze supporter sentiment during the … Continue reading
Posted in Testing
Tagged Architecture, Big Data, BigQuery, BQ, Cloud, Cloud Dataflow, ETL, Google, Google Cloud, Google Cloud Platform, Google Cloud Storage, Java, SDK
Leave a comment
Put On Your Streaming Shoes
The Kick-Off Meeting It went something along the lines of: Client: “We have a new requirement for you..” Shiners: “Shoot..” Client: “We’d like you to come up a solution that can insert 2 million rows per hour into a database and be able to deliver real-time … Continue reading
Posted in Javascript, Opinion
Tagged Big Data, BigQuery, Chart.js, Cloud, Distruptive, Google Cloud, Java, javascript, PHP, Software Development, Streaming
13 Comments
Shine @ AWS re: Invent 2012
I was lucky enough to be one of the 6,000 cloud geeks that descended on Vegas last week to attend AWS re:Invent 2012. This inaugural AWS developer conference was broken into 3 days. The first day was a bit of a warm-up day, with … Continue reading
Tagged aws, Cloud, Conference, re: Invent
1 Comment