Cloud Pub/Sub

Welcome to the Google Cloud Pub/Sub idea forum. You can submit and vote on ideas here to tell the Pub/Sub team which features you’d like to see.

This forum is for feature suggestions. If you’re looking for help forums, look here:

We can’t wait to hear from you!

  1. `endsWith` and `contains` should be included here - or pattern matching

    You might all this a feature request but I find it to be defective that you have a function to determine if an attribute "starts with" but not a function for "ends with". This makes it difficult to do things such as pick apart contexts from strings like

    com.mycompany.mydomain.myentity.myevent.v1

    For example, if I wanted to say "subscribe to all messages from mydomain" then hasPrefix(com.mycompany.mydomain) makes sense. But if I wanted to say "subscribe to any domain that publishes myevent" then I would need hasSuffix(myevent.v1).

    That said, I would also need something to determine something in the middle. For example "subscribe…

    6 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  2. Java process spawned when Pub/Sub emulator runs is not cleaned up

    When you start the Pub/Sub emulator (e.g. gcloud beta emulators pubsub start --project=myprog) there is also a java process which is spawned. I am assuming it contains a jetty process or similar.

    When you stop the Pub/Sub emulator (ctrl + c) this process remains. This causes issues in many ways. Some I have run into:

    1. The port in use is now locked while the java process remains. Starting Pub/Sub emulator again starts it on a new port.
    2. If there was an http push error, and the Pub/Sub emulator is continuously resending the message, it will continue to do so…
    4 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  3. Pub/Sub emulator huge delay on pull subscriptions

    I'm experiencing huge delays (which are blocking the development) from time to time on the pubsub emulator pull subscriptions.
    I'm starting the pubsub emulator via docker using latest google/cloud-sdk alpine image (I've tried also debian based image):

    FROM google/cloud-sdk:alpine
    RUN gcloud components install --quiet \
        pubsub-emulator \
        beta
    EXPOSE 8538
    VOLUME /opt/data
    

    Starting the docker with command:
    gcloud beta emulators pubsub start \
    --data-dir=/opt/data \
    --host-port=127.0.0.1:8538 \
    --host-port=0.0.0.0:8538 \
    --user-output-enabled \
    --log-http \
    --verbosity=debug

    I create topics and subscriptions via curl, I'm trying to create pull message subscriptions the same way.
    When I create pull message, there is sometimes 5…

    27 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  4. Fix the bug in emulator v331 where logging doesn't work

    I'm getting this error when starting the pubsub emulator (version 331):

    [pubsub] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
    [pubsub] SLF4J: Defaulting to no-operation (NOP) logger implementation
    [pubsub] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
    I'm running on Debian testing, with openjdk-11-{jre,jdk}:amd64 version 11.0.11+4-1.

    I tried installing packages: libslf4j-java and liblog4j1.2-java but it did not work.

    I found a hacky workaround, which I have described here on StackOverflow: https://stackoverflow.com/questions/66631222/pubsub-emulator-slf4j-error-on-start-no-logging/66631906

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  5. High CPU usage in the emulator

    The pub/sub emulator has a very high CPU usage, both during use but especially when idling. This appears to be more severe the more subscribers are added to a topic.

    The CPU usage is so high that it limits it's use case for local development, at least on weaker machines.

    As a suggestion I would propose to completely rewrite the emulator in Go, but I do understand that might be unfeasible.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  6. publicity entertainment Communication

    Nice Communication friendship publicity others

    2 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  7. 1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  8. 1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  9. Emulator fails when google-cloud-sdk path includes a space

    Launching the PubSub emulator on my workstation running MacOS 10.15.5 errors out immediately. Fortunately the error message is informative:

    $ gcloud beta emulators pubsub --verbosity debug start
    DEBUG: Running [gcloud.beta.emulators.pubsub.start] with arguments: [--verbosity: "debug"]
    DEBUG: Found Cloud SDK root: /Users/lex/Library/Application Support/google-cloud-tools-java/managed-cloud-sdk/LATEST/google-cloud-sdk
    Executing: /Users/lex/Library/Application Support/google-cloud-tools-java/managed-cloud-sdk/LATEST/google-cloud-sdk/platform/pubsub-emulator/bin/cloud-pubsub-emulator --host=localhost --port=8085
    [pubsub] Error: Unable to access jarfile /Users/lex/Library/Application

    Specifically, "Unable to access jarfile" message cuts off the path that it had correctly found (according to the previous line) right where the path includes a space. I haven't dug into the source, but it looks like the detected path has not been correctly escaped to handle…

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  10. Better description of the diagram describing the tasks clear and logically, incomes and outcomes for each of them. More relevant examples

    Start examples with generic steps to have a better understanding of the tool then provide more relevant examples (beyond Hello world!) through the steps instead just basing examples on do this and now this and after that because I found gaps and errors in multiple cases already

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  11. 1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  12. 2 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  13. Emulator needs to increase max gRPC message size to match production

    The emulator drops connections when publish messages exceed 4 MiB. However, the production pubsub service enforces a max publish message size limit of 10000000 bytes. It returns a friendly error code when you exceed that limit (at least up to some max; I haven't really tried). The two should match.

    The server needs to call maxInboundMessageSize() when it is being configured:

    https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerBuilder.html#maxInboundMessageSize-int-

    The exception logged in the emulator is the following:

    [pubsub] WARNING: Exception processing message
    [pubsub] io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304: 9000111
    [pubsub] at io.grpc.Status.asRuntimeException(Status.java:523)
    [pubsub] at io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:387)
    [pubsub] at io.grpc.internal.MessageDeframer.deliver(MessageDeframer.java:267)
    [pubsub] at io.grpc.internal.MessageDeframer.request(MessageDeframer.java:161)
    [pubsub] at…

    8 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  14. No esta traducida al español.

    No esta traducida al español.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  15. lenguaje

    No se cambia el idioma al seleccionar Español

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  16. Align pubsub and emulator message when topic not found

    When publishing to a topic that does not exist emulator and real pubsub return different message.
    Emulator:
    NOTFOUND: Topic not found
    Real PubSub:
    NOT
    FOUND: Resource not found

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  17. Subscription expiry with Cloud Functions Pub/Sub Trigger

    I've noticed an issue when a cloud function is triggered by pub/sub. When the function is deployed, a subscription is created but the expiry is set to 31 days. This means that if no messages are received for that time, the subscription disappears. After that the function appears to be deployed but will never do anything. Presumably the subscription should last as long as the function does, and be deleted if the function is removed.

    A further problem is that attempting to change the expiry time doesn't work. I tried:

    gcloud beta pubsub subscriptions update [subscription name removed] --expiration-period=never

    I…

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  18. Emulator should not need content-type header, or provide proper error response

    The real PubSub API request a publish request (potentially others) without the content-type header.

    The PubSub Emulator returns a 404 error if no content-type header is set, or a wrong content-type header is set.

    404 is the wrong HTTP status code in such a situation (400 is correct), and without a proper error message, this is extremely difficult to debug. I wasted a lot of time debugging whether I set up the topic correctly, and whether I constructed the URL correctly.

    Please either ignore the content-type header (like the PubSub API does), or provide a 400 response with a proper…

    5 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  19. Cloud pubsub emulator does not handle 202 response code correctly on push endpoints.

    Small discrepancy between the emulator and the production Cloud Pub/Sub.

    When responding to a push subscription with http code 202 the emulator will output the following message:

    [pubsub] INFO: Failed to push to endpoint '<endpoint here>': Fetched url '<url here>' but got a response code that is not an acknowledgement: 202

    Whereas per existing Cloud Pub/Sub documentation this should be considered a valid ack response for push endpoints:

    https://cloud.google.com/pubsub/docs/push

    (I haven't explicitely tested other response codes but I'm fairly certain that 204 and 200 are correctly handled, didn't test 201 and 102.)

    7 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  20. 1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1
  • Don't see your idea?

Cloud Pub/Sub

Categories

Feedback and Knowledge Base