Cloud Functions V2 API - Unable to use 8192 or 4096 on gen1 PATCH

[Peter_Roberts]

Hi all,

We started using the V2 cloud functions API to deploy all our gen1 & gen2 cloud functions, via the google-api-python-client wrapper/client.

We have tooling that deploys cloud functions regardless of gen type, we simply GET the current cloud function and PATCH to trigger redeployment.

Things are going good, but noticed we cannot deploy any gen1 cloud functions with 4096 or 8192 of memory, but lower memory selections work fine.

I was recommended to use the v2 API for gen2 functions, and v1 API for gen1 cloud functions, but i thought the V2 API supports both gen types now..... would be nice to use the v2 API for gen1 & gen2 cloud function deployments for all memory types.

Example:

Unable to use 8192M in PATCH to https://cloudfunctions.googleapis.com/v2/{function.name} (gen1 cloud function). Below snippet body i used in my PATCH, that does not work(other supported memory's do work though)

{
  'name': 'projects/myproj/locations/europe-west2/functions/my-function',
  'serviceConfig': {
    'availableMemory': '8192M'
  },
  'environment': 'GEN_1'
}

Error/response(added new lines for readability)

googleapiclient.errors.HttpError: <HttpError 400 when requesting https://cloudfunctions.googleapis.com/v2/projects/myproj/locations/europe-west2/functions/my-function?alt=json returned
"Failed to update 1st Gen function projects/myproj/locations/europe-west2/functions/my-function memory_mb: Invalid memory allocation.
Should be one of: [128, 256, 512, 1024, 2048, 4096, 8192]".

I tried using 8192Mi, which was accepted but defaulted to 256M after deployment.
Deploying the gen1 cloud function via the GCP console, or gcloud CLI, work perfect.

I think v2 API left beta maybe recently'ish, so maybe i should hold off using v2 API for gen1 cloudfunctions for a while longer.

Refs:

• https://googleapis.github.io/google-api-python-client/docs/dyn/cloudfunctions_v2.projects.locations.... 
• https://cloud.google.com/functions/docs/reference/rest/v2/projects.locations.functions/patch 
• I also raised this, as not sure if this is a bug: https://issuetracker.google.com/issues/342557692 

[apeltop]

Hi Peter Roberts.

If you look at the Cloud SDK Reference, you can see the differences between Gen1 and Gen2.

To create an 8192M Gen1 function. You should use 8192MB instead of 8192M.

Limit on the amount of memory the function can use.
Allowed values for v1 are: 128MB, 256MB, 512MB, 1024MB, 2048MB, 4096MB, and 8192MB.

Allowed values for GCF 2nd gen are in the format: <number><unit> with allowed units of "k", "M", "G", "Ki", "Mi", "Gi". Ending 'b' or 'B' is allowed.

Examples: 100000k, 128M, 10Mb, 1024Mi, 750K, 4Gi.

By default, a new function is limited to 256MB of memory. When deploying an update to an existing function, the function keeps its old memory limit unless you specify this flag.

I think it would be helpful to have more information about the part that is set to 8192Mi and then set to 256M when after deployed.

Ref

https://cloud.google.com/sdk/gcloud/reference/functions/deploy#--memory

[Peter_Roberts]

Thanks @apeltop for responding.

No joy with that im afraid.
I was curious if this relates to 'CPU:RAM' ratio when >4gb......but maybe not, im just guessing at this point. 

(below is the same for 4096M variants, but anything <4096 is good/success) 

# 8192MB
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://cloudfunctions.googleapis.com/v2/projects/myproj/locations/europe-west2/functions/my-functio...
returned "Invalid Request. Violations: Invalid memory allocation 8192MB specified for 1st Gen function.". Details: "Invalid Request. Violations: Invalid memory allocation 8192MB specified for 1st Gen function.">

# 8192MiB
Same as above 

# 8192M
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://cloudfunctions.googleapis.com/v2/projects/myproj/locations/europe-west2/functions/my-functio...
returned "Failed to update 1st Gen function projects/myproj/locations/europe-west2/functions/my-function: memory_mb: Invalid memory allocation. Should be one of: [128, 256, 512, 1024, 2048, 4096, 8192]". 

# 8192Mi
PATCH is accepted, but after deployment completes, the function has 256M

Script to repro:

#!/usr/bin/env python3
"""-------------------------------------------------------------------------------------------------------
 pip install google-api-python-client
 python test_gen1_via_v2_api.py
-------------------------------------------------------------------------------------------------------"""
from googleapiclient.discovery import build as gcp_apis_client


def get_cloud_function_client(api_version: str) -> gcp_apis_client:
    """
    Return GCP API client, cloudfunctions

    :return: cloudfunctions client
    """
    cloudfunction_client: gcp_apis_client = gcp_apis_client("cloudfunctions",
                                                            api_version)
    return cloudfunction_client


def get_cloud_function(cloud_function_client: gcp_apis_client,
                       cloud_function_name: str,
                       cloud_function_region: str,
                       cloud_function_project: str) -> dict:
    """
    GET a cloud function.

    :param gcp_apis_client cloud_function_client: An instance of the cloudfunctions python API client/wrapper
    :param str cloud_function_name: The name of the cloud function
    :param str cloud_function_region: The region the cloud function is hosted in
    :param str cloud_function_project: The GCP project the cloud function is hosted in
    :return: The json object of the cloud function
    """
    current_cloud_function_details = cloud_function_client.projects().locations().functions().get(
        name="projects/%s/locations/%s/functions/%s" % (cloud_function_project,
                                                        cloud_function_region,
                                                        cloud_function_name)).execute()

    return current_cloud_function_details


def patch_cloud_function(cloud_function_client: gcp_apis_client,
                         cloud_function_name: str,
                         cloud_function_region: str,
                         cloud_function_project: str,
                         current_cloud_function_details: dict) -> dict:
    """
    Method to PATCH a cloud function to trigger a redeployment of it.

    :param gcp_apis_client cloud_function_client: An instance of the cloudfunctions python API client/wrapper
    :param str cloud_function_name: The name of the cloud function
    :param str cloud_function_region: The region the cloud function is hosted in
    :param str cloud_function_project: The GCP project the cloud function is hosted in
    :param dict current_cloud_function_details: The current cloud function details
    :return: The response
    """
    patched_resp = cloud_function_client.projects().locations().functions().patch(
        name="projects/%s/locations/%s/functions/%s" % (cloud_function_project,
                                                        cloud_function_region,
                                                        cloud_function_name),
        body=current_cloud_function_details).execute()

    return patched_resp


def main():
    """
    PATCH gen1 cloud function, via v2 API, using different memory's
    """
    # TODO: First, manually deploy a gen1 cloud function(python runtime in this example).

    # Client
    cloud_function_client = get_cloud_function_client(api_version="v2")

    # GET current cloud function
    cloud_function_name = "my-function"
    cloud_function_region = "europe-west2"
    cloud_function_project = "myproj"
    current_cloud_function = get_cloud_function(cloud_function_client,
                                                cloud_function_name,
                                                cloud_function_region,
                                                cloud_function_project)

    # Update memory
    current_cloud_function["serviceConfig"]["availableMemory"] = "8192MB"      # Failure
    # current_cloud_function["serviceConfig"]["availableMemory"] = "8192M"     # Failure
    # current_cloud_function["serviceConfig"]["availableMemory"] = "8192Mi"    # Accepted, but deploys as 256M
    # current_cloud_function["serviceConfig"]["availableMemory"] = "8192MiB"   # Failure

    # current_cloud_function["serviceConfig"]["availableMemory"] = "4096MB"    # Failure
    # current_cloud_function["serviceConfig"]["availableMemory"] = "4096M"     # Failure
    # current_cloud_function["serviceConfig"]["availableMemory"] = "4096Mi"    # Accepted, but deploys as 256M
    # current_cloud_function["serviceConfig"]["availableMemory"] = "4096MiB"   # Failure

    # current_cloud_function["serviceConfig"]["availableMemory"] = "2048M"     # Success
    # current_cloud_function["serviceConfig"]["availableMemory"] = "1024M"     # Success
    # current_cloud_function["serviceConfig"]["availableMemory"] = "512M"      # Success
    # current_cloud_function["serviceConfig"]["availableMemory"] = "256M"      # Success
    # current_cloud_function["serviceConfig"]["availableMemory"] = "128M"      # Success

    # PATCH
    patched_resp = patch_cloud_function(cloud_function_client,
                                        cloud_function_name,
                                        cloud_function_region,
                                        cloud_function_project,
                                        current_cloud_function)
    print(patched_resp)


if __name__ == "__main__":
    main()

V2 API cloud functions object

{
  'name': 'projects/myproj/locations/europe-west2/functions/my-function',
  'buildConfig': {
    'build': 'projects/my-proj-number/locations/europe-west2/builds/redacted',
    'runtime': 'python312',
    'entryPoint': 'test_entrypoint',
    'source': {
      'storageSource': {
        'bucket': 'my-bucket',
        'object': 'cloud_function_zips/my-function.zip'
      }
    },
    'dockerRegistry': 'ARTIFACT_REGISTRY',
    'automaticUpdatePolicy': {}
  },
  'serviceConfig': {
    'timeoutSeconds': 60,
    'maxInstanceCount': 3000,
    'ingressSettings': 'ALLOW_ALL',
    'uri': 'https://europe-west2-myprof.cloudfunctions.net/my-function',
    'serviceAccountEmail': 'myproj@appspot.gserviceaccount.com',
    'availableMemory': '2048M',
    'revision': '48',
    'maxInstanceRequestConcurrency': 1,
    'securityLevel': 'SECURE_ALWAYS'
  },
  'state': 'ACTIVE',
  'updateTime': '2024-05-28T17:36:33.712275837Z',
  'labels': {
    'deployment-tool': 'cli-gcloud',
  },
  'environment': 'GEN_1',
  'url': 'https://europe-west2-myproj.cloudfunctions.net/my-function'
}

V1 cloud functions object

{
  'name': 'projects/myproj/locations/europe-west2/functions/my-function',
  'sourceArchiveUrl': 'gs://my-bucket/cloud_function_zips/my-function.zip',
  'httpsTrigger': {
    'url': 'https://europe-west2-myproj.cloudfunctions.net/my-function',
    'securityLevel': 'SECURE_ALWAYS'
  },
  'status': 'ACTIVE',
  'entryPoint': 'test_entrypoint',
  'timeout': '60s',
  'availableMemoryMb': 2048,
  'serviceAccountEmail': 'myproj@appspot.gserviceaccount.com',
  'updateTime': '2024-05-28T17:36:22.291Z',
  'versionId': '48',
  'labels': {
    'deployment-tool': 'cli-gcloud'
  },
  'runtime': 'python312',
  'maxInstances': 3000,
  'ingressSettings': 'ALLOW_ALL',
  'buildId': 'redacted',
  'buildName': 'projects/my-proj-number/locations/europe-west2/builds/redacted',
  'dockerRegistry': 'ARTIFACT_REGISTRY',
  'automaticUpdatePolicy': {}
}

[Peter_Roberts · Accepted solution]

Hi all,

Good news

I was able to PATCH 8192M and 4096M successfully. So maybe Google/GCP updated the V2 cloudfunctions API for gen1 cloud functions recently. 

Ill update the bug/ticket also, and ill resolve this community post with the update/findings from GCP/google https://issuetracker.google.com/issues/342557692 

current_cloud_function["serviceConfig"]["availableMemory"] = "8192M"
current_cloud_function["serviceConfig"]["availableMemory"] = "4096M"