Yesterday, we announced that Google Cloud Platform big data services are taking a big step forward by allowing everyone to use big data the cloud way. Google BigQuery has many new features and is now available in European zones. These improvements were designed to extend BigQuery’s performance and capabilities to give you greater peace-of-mind and control over your data.
European Data Location Control
You now have the option to store your BigQuery data in European locations while continuing to benefit from a fully managed service, now with the option of geographic data control, without low-level cluster maintenance headaches. Feel free to contact the Google Cloud Platform technical support team for details on how to set this up.
One of BigQuery’s most popular features is the ability to stream data into the service for real-time analysis. To allow such low-latency analysis on very high-volume streams, we’ve increased the default insert-rate limit from 10,000 rows per second, per table, to 100,000 rows per second, per table. In addition, the row-size limit has increased from 20 KB to 1 MB, and pricing will move from a per-row model to a per-byte model for better flexibility and scale.
BigQuery can now tackle a wider range of enterprise applications with the addition of data expiration controls and row-level permissions. Row-level permissions eliminate the need to create different views for different users, allowing secure shared access to systems such as finance or HR. This ensures that you get the information that’s relevant to you. In addition, data in BigQuery will be encrypted at rest.
Google Cloud Platform Logging Integration
Google Cloud Logging provides a powerful set of tools for managing your operations and understanding the systems powering your business; now, it also lets your Google App Engine and Google Compute Engine applications stream their logs into BigQuery. This allows you to perform real-time analysis on your log data and gain insight into how your system is performing and how your users are behaving. By joining application logs with your marketing and partnership data, you can rapidly evaluate the effectiveness of your outreach, or apply context from user profile info into your application logs to quickly assess what behavior resulted from specific customer interactions, providing easy and immediate value to both system administrators and business analysts.
Frequently requested features
Additionally, we’ve implemented a number of new features you’ve been asking for. You can now:
- load content from Google Cloud Datastore
- nest query results
- leverage FULL and RIGHT OUTER JOINS
- roll up your aggregations to include subtotals
- recover data from recently deleted tables
- take advantage of a number of improvements to the web UI
For a full list of features, take a look at the release notes.
BigQuery continues to provide exceptional scale and performance without requiring you to deploy, augment or update your own clusters. Instead, you can focus on getting meaningful insights from massive amounts of data. For example:
- BigQuery absorbs real-time streams of customer data totaling more than 100 TB per day, which you can query immediately. All this data is in addition to the hundreds of terabytes loaded daily from other sources. If you have fast-moving, large-scale applications such as IoT, you can now make quick, accurate decisions against in-flight applications.
- We have customers currently running queries that scan multiple petabytes of data or tens of trillions of rows using a simple SQL query, without ever having to worry about system provisioning, maintenance, fault-tolerance or performance tuning.
With BigQuery’s new features, you can analyze even more data and access it faster than before, in brand new ways. To get started, learn more about BigQuery, read the documentation, and try it out for yourself.
-Posted by Andrew Kowal, Product Manager
Feed Source: Google Cloud Platform Blog
Article Source: Take your big data to new places with Google BigQuery