gcloud endpoints services deploy
「Google Cloud Platform（GCP）へのワークロードの移行過程では、複数のデータセンター間で通信のセキュリティを強化することが必要でした。ファイアウォールや場当たり的な認証といった旧来の方法では、あっという間に ACL がぐちゃぐちゃになってしまい、とても対応できません。一方、Cloud Endpoints は標準化された認証システムを提供してくれました。」 — Laurie Clark-Michalek 氏、Qubit のインフラストラクチャ エンジニア
coolcloudapi.googleapis.com/v1/coolthings/12301221312132
import "google/protobuf/empty.proto"; // A simple Bookstore API. // // The API manages shelves and books resources. Shelves contain books. service Bookstore { // Returns a list of all shelves in the bookstore. rpc CreateShelf(CreateShelfRequest) returns (Shelf) {} // Returns a specific bookstore shelf. rpc GetShelf(GetShelfRequest) returns (Shelf) {} } // A shelf resource. message Shelf { // A unique shelf id. int64 id = 1; // A theme of the shelf (fiction, poetry, etc). string theme = 2; } // Request message for CreateShelf method. message CreateShelfRequest { // The shelf resource to create. Shelf shelf = 1; } // Request message for GetShelf method. message GetShelfRequest { // The ID of the shelf resource to retrieve. int64 shelf = 1; }
type: google.api.Service config_version: 3 name: bookstore.endpoints..cloud.goog title: Bookstore gRPC API apis: - name: endpoints.examples.bookstore.Bookstore Http: rules: # 'CreateShelf' can be called using the POST HTTP verb and the '/shelves' URL # path. The posted HTTP body is the JSON respresentation of the 'shelf' field # of 'CreateShelfRequest' protobuf message. # # Client example: # curl -d '{"theme":"Music"}' http://DOMAIN_NAME/v1/shelves # - selector: endpoints.examples.bookstore.Bookstore.CreateShelf post: /v1/shelves body: shelf # # 'GetShelf' is available via the GET HTTP verb and '/shelves/{shelf}' URL # path, where {shelf} is the value of the 'shelf' field of 'GetShelfRequest' # protobuf message. # # Client example - returns the first shelf: # curl http://DOMAIN_NAME/v1/shelves/1 # - selector: endpoints.examples.bookstore.Bookstore.GetShelf get: /v1/shelves/{shelf}
# # Request authentication. # authentication: providers: - id: google_service_account # Replace SERVICE-ACCOUNT-ID with your service account's email address. issuer: SERVICE-ACCOUNT-ID jwks_uri: https://www.googleapis.com/robot/v1/metadata/x509/SERVICE-ACCOUNT-ID rules: # This auth rule will apply to all methods. - selector: "*" requirements: - provider_id: google_service_account
blogs/endpointslambda/aeflex-endpoints/
@app.route('/processmessage', methods=['POST']) def process(): """Process messages with information about S3 objects""" message = request.get_json().get('inputMessage', '') # add other processing as needed # for example, add event to PubSub or # download object using presigned URL, save in Cloud Storage, invoke ML APIs return jsonify({'In app code for endpoint, received message': message})
host: "echo-api.endpoints.aeflex-endpoints.cloud.goog"
inputMessage
# This section configures the processmessage endpoint. "/processmessage": post: description: "Process the given message." operationId: "processmessage" produces: - "application/json" responses: 200: description: "Return a success response" schema: $ref: "#/definitions/successMessage" parameters: - description: "Message to process" in: body name: inputMessage required: true schema: $ref: "#/definitions/inputMessage" security: - api_key: [] definitions: successMessage: properties: message: type: string inputMessage: # This section contains information about the S3 bucket and object to be processed. properties: Bucket: type: string ObjectKey: type: string ContentType: type: string ContentLength: type: integer ETag: type: string PresignedUrl: type: string
gcloud service-management deploy openapi.yaml
Service Configuration [2017-03-05r2] uploaded for service "echo-api.endpoints.aeflex-endpoints.cloud.goog"
endpoints_api_service: # The following values are to be replaced by information from the output of # 'gcloud service-management deploy openapi.yaml' command. name: echo-api.endpoints.aeflex-endpoints.cloud.goog config_id: 2017-03-05r2
gcloud app deploy
blogs/endpointslambda/lambdafunctioninline.py.
from __future__ import print_function import boto3 import json import os import urllib import urllib2 print('Loading function') s3 = boto3.client('s3') endpoint_api_key = os.environ['ENDPOINT_API_KEY'] endpoint_url = "https://aeflex-endpoints.appspot.com/processmessage" def lambda_handler(event, context): # Get the object information from the event bucket = event['Records'][0]['s3']['bucket']['name'] object_key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key'].encode('utf8')) try: # Retrieve object metadata response = s3.head_object(Bucket=bucket, Key=object_key) # Generate pre-signed URL for object presigned_url = s3.generate_presigned_url('get_object', Params = {'Bucket': bucket, 'Key': object_key}, ExpiresIn = 3600) data = {"inputMessage": { "Bucket": bucket, "ObjectKey": object_key, "ContentType": response['ContentType'], "ContentLength": response['ContentLength'], "ETag": response['ETag'], "PresignedUrl": presigned_url } } headers = {"Content-Type": "application/json", "x-api-key": endpoint_api_key } # Invoke Cloud Endpoints API request = urllib2.Request(endpoint_url, data = json.dumps(data), headers = headers) response = urllib2.urlopen(request) print('Response text: {} \nResponse status: {}'.format(response.read(), response.getcode())) return response.getcode() except Exception as e: print(e) print('Error integrating lambda function with endpoint for the object {} in bucket {}'.format(object_key, bucket)) raise e
「Google Cloud Platfrom への移行時に、複数のデータセンター間でセキュアに通信を行う必要がありました。ファイアウォールやアドホックな認証では、すぐに ACL がぐちゃぐちゃになってとても対応できません。一方 Cloud Endpoints は、Google の積み重ねてきたセキュリティに支えられた、標準化された認証システムを提供してくれました。」 — Laurie Clark-Michalek, Infrastructure Engineer at Qubit
