The Google Developers Console and App Engine

The Google Developers Console lets you manage your App Engine application along with other Google Cloud Platform resources. Once you have a Google account, you may start using this console. Access the console by visiting console.developers.google.com and signing in.

You can use the console to monitor your application's performance, and control various serving and configuration settings, depending on your user role permits. Visit the console pages devoted to App Engine by expanding the Compute item in the left-side menu, and the App Engine item beneath it.

Using the Developers Console

The Developers Console is used to manage all aspects of your projects, applications, and resources including:

Manage a Cloud Platform project

• Creating a project
• Project members, service accounts, deleting a project
• Billing accounts
• Cookies, Authentication and Log retention

Manage an App Engine application

• Snapshots of current status
• Status of running instances
• Manage versions and modules
• Traffic splitting
• View logs
• Security scan

Manage application resources

• Quota usage
• Taskqueues
• Memcache
• Datastore

The Developers Console lets you manage many other Google Cloud Platform features. Learn more.

The App Engine dashboard

The dashboard page is available in the left-side menu under Compute>App Engine>Dashboard.

App Engine applications are composed of one or more modules. Each module handles a particular set of HTTP requests. As your application evolves, you may deploy more than one version of a particular module, and establish routing rules so different requests may target different versions of the same module.

At the top of the dashboard, a pair of pulldown menus sets the scope of the information to be displayed. You can select:

• All modules and versions
• A single module with all of its versions
• A single module with one of its versions

A link appears to the right of these pulldowns. It will target your app to the level of detail that you've selected (using the default module and/or version as appropriate).

Usage graphs

A graph appears below the module/version selectors. Use the pulldown menu at the top left to select a performance metric or resource. Select a time period (from 1 hour to 20 days) at the top right, then select one of the graph types:

Current status tables

Five tables appear below the dashboard graph. They summarize the current status of the module(s) and version(s) that you've specified at the top of the page:

You can sort the tables on most of the columns. When you place the mouse to the right of a column title a caret appears. Click on it to select increasing/decreasing sort order.

Instances

The instance page is located in the left-side menu under Compute>App Engine>Instances.

The top of the page is similar to the dashboard. Use the pulldown menus to select a specific module and version. The same Dashboard graphs can be seen on this page.

Below the graph, a table lists every instance in the scope you selected. If the selected module is running in a managed VM, a column indicates if the VM is being managed by Google or the user. The other columns in the table show:

• The average Queries Per Second and latency (ms) over the last minute
• The number of requests received in the last minute
• The number of errors reported in the last minute
• Current memory usage (MB)
• The start time for the instance, indicating its age - how it's been running.
• A link to the logs viewer for the instance
• The instance's availability type; resident or dynamic.

Versions

The Versions page is available in the left-side menu under Compute>App Engine>Versions.

Use the pulldown menu to select a specific module. All of the module's deployed versions will appear in a table below, showing:

• The version name
• The version size
• The version runtime
• The number of instances of each version
• The deployment time

If you have owner or edit permissions you can select a version and make it the default version. You can also delete a version, and configure traffic splitting or traffic migration, using the buttons below the table. They are only enabled when you have selected the default module.

Traffic Splitting

Traffic splitting takes incoming requests to your application’s default module and splits them among the default version and one or more other versions, according to a distribution that you specify. Traffic splitting allows you to roll out features for your app slowly over a period of time. It can also be used for A/B Testing. When you deploy a new version of your default module, you can use traffic migration to re-route requests gradually between the existing default version and the newly deployed version.

You must choose whether to split the traffic by IP address or an HTTP cookie. It's easier to set up an IP address split, but a cookie split is more precise.

IP Address Splitting

When the application receives a request, it hashes the IP address to a value between 0–999, and uses that number to route the request. IP address splitting has some significant limitations:

• Because IP addresses are independently assigned to versions, the resulting traffic split will differ somewhat from what you specify. For example, if you ask for 5% of traffic to be delivered to an alternate version, the actual percent of traffic the the version might be between 3–7%. The more traffic your application receives, the more accurate the split will be.
• IP addresses are reasonably sticky, but are not permanent. Users connecting from cell phones may have a shifting IP address throughout a single session. Similarly, a user on a laptop may be moving from home to a cafe to work, and hence also shifting through IP addresses. As a result, the user may have an inconsistent experience with your app as their IP address changes.
• Requests sent to your app from outside of Google's cloud infrastructure will work normally. However, requests sent from your app to another app (or to itself) are not sent to a version because they all originate from the a small number of IP addresses. This applies to any traffic between apps within Google's cloud infrastructure. If you need to send requests between apps, use cookie splitting instead.

Cookie Splitting

The application looks in the HTTP request header for a cookie named GOOGAPPUID that contains a value between 0–999. If the cookie exists the number is used to route the request. If there is no such cookie the request is routed randomly - and when the response is sent the app adds a GOOGAPPUID cookie, with a random value between 0–999. (The cookie is added only when traffic split by cookie is enabled and the response does not already contain a GOOGAPPUID cookie.)

Splitting traffic using cookies makes it easier to accurately assign of users to versions, which allows more precision in traffic routing (as small as 0.1%). Cookie splitting also has some limitations:

• If you are writing a mobile app or running a desktop client, it needs to manage the GOOGAPPUID cookies. When your app server sends back a response with a Set-Cookie header, you must store the cookie and include it with each subsequent request. (Browser-based apps already manage cookies in this way automatically.)

• Splitting internal requests requires extra work. Requests sent server-side from your app to another app (or to itself) can be sent to a version, but doing so requires that you forward the user's cookie with the request. Note that internal requests that don't originate from the user are not recommended for versions.

Caching and traffic splitting

Caching issues can exist for any App Engine application, especially when deploying a new version. Traffic splitting often makes subtle caching problems more apparent.

For example, assume you are splitting traffic between two versions, A and B, and some external cacheable resource (like a css file) changed between versions. Now assume the client makes a request, and the response contains an external reference to the cached file. The local HTTP cache will retrieve the file if it's in the cache - no matter which version of the file is cached and which version of the application served up the response. The cached resource could be incompatible with the data in the response.

You can avoid caching problems for dynamic resources by setting the Cache-Control and Expires headers. These headers tell proxies that the resource is dynamic. It is best to set both headers, since not all proxy servers support the HTTP/1.1 Cache-Control header properly. If you want more information on caching in general, try these web pages:

• Header Fields in the HTTP/1.1 RFC
• Document Caching in the Browser Security Handbook

For cacheable static resources that vary between versions, you can change the resource's URL between versions. If each version of a static resource is serving from a different URL, the versions can happily co-exist in proxy servers and browser caches.

If the app sets the Vary: Cookie header, the uniqueness of a resource is computed by combining the cookies and the URL for the request. This approach increases the burden on cache servers. There are 1000 possible values of GOOGAPPUID, and hence 1000 possible entries for each URL for your app. Depending on the load on the proxies between your users and your app, this may decrease the cache hit rate. Also note that for the 24 hours after adding a new batch of users to a version, they may still see cached resources. However, using Vary: Cookie can make it easier to rename static resources that are changing between versions.

The Vary: Cookie technique doesn't work in all circumstances. In general, if your app is using cookies for other purposes, you must consider how this affects the burden on proxy servers. If codeninja had its own cookie that had 100 possible values, then the space of all possible cache entries becomes a very big number (100 * 1000 = 100,000). In the worst case, there is a unique cookie for every user. Two common examples of this are Google Analytics (__utma) and SiteCatalyst (s_vi). In these cases, every user gets a unique copy, which severely degrades cache performance and may also increase the billable instance hours consumed by your app.

Task queues

Using task queues, a module handling requests can pass off work to a background task that executes asynchronously.

The Task Queues page is available in the left-side menu under Compute>App Engine>Task queues. This page displays information about the two kinds of task queues, pull queues, push queues, and also tasks that belong to cron jobs. Select the type of task using the menu bar at the top of the page. When you are viewing push and pull queue reports, a quotas link appears at the upper right that hides and shows your app's use of task queue quotas.

A table lists all of your application's queues, filtered by the type you selected. Click on a queue name to show the tasks scheduled to run that queue. Click on a task name to see detailed information about the task's payload, headers, and previous run (if any).

You can manually delete individual tasks or purge every task from a queue. This is useful if a task cannot be completed successfully and is stuck waiting to be retried. You can also pause and resume a queue.

Security scans

The Security Scanner page is available in the left-side menu under Compute>App Engine>Security scans.

The Google Cloud Security Scanner identifies security vulnerabilities in your Google App Engine web applications. It crawls your application, following all links within the scope of your starting URLs, and attempts to exercise as many user inputs and event handlers as possible, in order to discover vulnerabilities.

In order to use the security scanner you must be an owner of the project. For more information, read the security scanner instructions.

Quota details

The Quota details page is available in the left-side menu under Compute>App Engine>Quota details. This page shows daily usage and quota consumption for your project. Resources are grouped by API. If some project resource exceeds 50% of its quota halfway through the day, it may exceed the quota before the day is over. For more information about quotas, see the App Engine quota page and Why is My App Over Quota?

You can also use the billing export feature to write daily Google Cloud Platform usage and cost estimates to a CSV or JSON file that's stored in a Google Cloud Storage bucket you specify.

Memcache

App Engine provides a distributed in-memory data cache that can improve an application's performance. The cache contains key/value pairs, and the actual pairs in memory at any time change as items are written and retrieved from the cache.

The Memcache page is available in the left-side menu under Compute>App Engine>Memcache. The top of the page displays key information about the state of your project's memcache:

• Memcache service level (shared or dedicated memcache). The shared level is free, and is provided on a "best effort" basis with no space guarantees. The dedicated class assigns resources exclusively to your application and is billed by the gigabyte-hour of cache space (it requires billing to be enabled). If you are an owner of the project, you can switch between the two service levels.

• Hit ratio (as a percentage) and the raw number of memcache hits and misses.

• Number of items in the cache.

• The age of the oldest cached item. Note that the age of an item is reset every time it is used, either read or written.

• Total cache size.

Managing keys

You can add a new key to the cache, and search for an existing key.

Memcache compute units

Memcache use is often measured in terms of operations per second per GB of cache space. An alternate way to measure cache performance is by memcache compute units or MCUs. MCUs are computed separately for memory set and get operations and depend on the size of the value associated with a key:

The MCU for a set depends on the value size. It is 2 times the cost of a successful get-hit operation. Other operations are assigned MCU as follows:

Top keys

The memcache page displays a list of the 20 top keys by MCU over the past hour. This can help identify problematic "hot keys". The list is created by sampling API calls; only the most frequently accessed keys are tracked. Although the viewer displays 20 keys, more may have been tracked. The list gives each key's relative operation count as a percentage of all memcache traffic. If an application is a heavy user of memcache and some keys are used very often, the display may include warning indicators.

Search

App Engine provides a Search service that stores structured documents in one or more indexes. A document contains data-typed fields. You can search an index for documents with fields that match queries.

The Search page is available in the left-side menu under Compute>App Engine>Search. On this page you can view all the indexes in a project and the documents contained in each index. You can also run a query on a selected index.

Settings

The Settings page is located in the left-side menu under Compute>App Engine>Settings. It has two tabs, one for application settings, and another to add custom domains for you app.

Application Settings

If you have owner permissions, you can use this section to set:

• The Google login cookie expiration interval. Cookies can expire in one day, one week, or two weeks.

• The Google authentication method. If you do not to authenticate against a Google Apps domain, you can switch between a Google Accounts API or Federated Login.

• The logs retention criteria (storage limit and time interval). By default, application logs are stored free of charge for up to 90 days and with a 1 GB maximum size limit. if you enable billing you can increase the size limit and extend the time limit up to one year. The cost for log storage above the free quota is $0.026 per gigabyte per month.

Custom Domains

App Engine offers several options for setting up custom domains, subdomains, and aliases for your application. If desired, your app can use SSL security. It can also, for instance, be served along with other applications at Google Apps. To learn more about these options or to set up a custom domain for your app, see Using a Custom Domain.

Using other console pages

A Google App Engine application may use services and features that belong to other Google Cloud Platform products. These services are found elsewhere in the Developer console, outside of the Compute->App Engine section.

Creating a project

The Developers Console help page contains instructions for creating and deleting a project.

Billing

Use the gear-shaped pull down menu in the upper right corner of the console to access billing information:

• To manage billing for a project, select Project billing settings
• To manage your billing accounts select Billing accounts

Permissions

Use the Permissions page to manage members and service accounts, and to delete projects.

Project members

A project member can be assigned one of three roles. Each role permits a different level of access to the project and its application and resources. For App Engine applications, the role also controls the permissible actions in the glcoud and appcfg command line tools, that are used to deploy and manage applications.

Service Accounts

If your application needs to communicate with other Google Cloud Platform services, you may need to use the Permissions page to establish a service account.

Deleting an application

You can delete a project you own by selecting the project on the permissions page and clicking Remove. This will delete the project, all application modules and versions in the project, and all Google Cloud Platform services associated with the project.

Logs

The logs for all of a project's services, including App Engine app logs, are found on the Logs page under Monitoring>Logs

Datastore

App Engine's Datastore sevice provides schemaless, scalable storage of structured data objects. Use the pages in the Storage>Cloud Datastore section to manage your application's datastore.

Datastore dashboard

The Cloud Datastore Dashboard shows you data for the entities in your application's Datastore, as well as statistics for the built-in and composite indexes. The Statistics page displays data in various ways:

• A pie chart that shows datastore space used by each property type (string, double, blob, etc.).

• A pie chart showing datastore space by entity kind.

• A table with the total space used by each property type. The "Metadata" property type represents space consumed by storing properties inside an entry that is not used by the properties directly. The "Datastore Stats" entity, if any, shows the space consumed by the statistics data itself in your datastore.

• A table showing total size, average size, entry count, and the size of all entities, and the built-in and composite indexes.

By default, the pie charts display statistics for all entities. You can restrict the pie charts to a particular entity kind by choosing from the drop- down menu.

The statistics data is stored in your app's datastore. To make sure there's room for your app's data, Google App Engine will only store statistics if they consume less than 10 megabytes if namespaces are not used, or 20 megabytes if namespaces are used. If your app's stats go over the limit, kind-specific stats are not emitted. If subsequently these "non-kind" stats exceed the limits, then no stats are updated until stats storage drops below the size limitation. (Any previously reported statistics will remain.) You can determine whether this is happening by looking at the timestamps on stat records. If the timestamp gets older than 2 days, and stats in other applications are being updated regularly then this can indicate that you have run into stat storage size limits for your app.

The space consumed by the statistics data increases in proportion to the number of different entity kinds and property types used by your app. The more different entities and properties used by your app, the more likely you are to reach the stat storage limit. Also, if you use namespaces, remember that each namespace contains a complete copy of the stats for that namespace.

Indexes

The Indexes page shows a table of all indexes. You can create a new entity in this page.

Query

The Query page lets you select an entity kind and construct a query by applying filters. You can also create a new entity in this page.

Backup/restore, copy, delete

The Developers Console does not yet support datastore management. You should continue to use the Admin Console's Datastore Admin page for these functions.