Cloud Pub/Sub

Welcome to the Google Cloud Pub/Sub idea forum. You can submit and vote on ideas here to tell the Pub/Sub team which features you’d like to see.

This forum is for feature suggestions. If you’re looking for help forums, look here:

We can’t wait to hear from you!

  1. Emulator needs to increase max gRPC message size to match production

    The emulator drops connections when publish messages exceed 4 MiB. However, the production pubsub service enforces a max publish message size limit of 10000000 bytes. It returns a friendly error code when you exceed that limit (at least up to some max; I haven't really tried). The two should match.

    The server needs to call maxInboundMessageSize() when it is being configured:

    https://grpc.github.io/grpc-java/javadoc/io/grpc/ServerBuilder.html#maxInboundMessageSize-int-

    The exception logged in the emulator is the following:

    [pubsub] WARNING: Exception processing message
    [pubsub] io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304: 9000111
    [pubsub] at io.grpc.Status.asRuntimeException(Status.java:523)
    [pubsub] at io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:387)
    [pubsub] at io.grpc.internal.MessageDeframer.deliver(MessageDeframer.java:267)
    [pubsub] at io.grpc.internal.MessageDeframer.request(MessageDeframer.java:161)
    [pubsub] at…

    8 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  2. undelivered messages Retains period too low.

    clients cam have vacations up to 6 weeks sometimes. having 7 day retention for those messages too low.

    7 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  3. 7 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  4. Cloud pubsub emulator does not handle 202 response code correctly on push endpoints.

    Small discrepancy between the emulator and the production Cloud Pub/Sub.

    When responding to a push subscription with http code 202 the emulator will output the following message:

    [pubsub] INFO: Failed to push to endpoint '<endpoint here>': Fetched url '<url here>' but got a response code that is not an acknowledgement: 202

    Whereas per existing Cloud Pub/Sub documentation this should be considered a valid ack response for push endpoints:

    https://cloud.google.com/pubsub/docs/push

    (I haven't explicitely tested other response codes but I'm fairly certain that 204 and 200 are correctly handled, didn't test 201 and 102.)

    7 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  5. Python client library as on today is not working in an environment where http_proxy is not set

    Following error is occuring:

    E0624 07:18:51.447702419 9603 uriparser.cc:46] bad uri.scheme: ''
    E0624 07:18:51.447727494 9603 uri
    parser.cc:52] ^ here
    E0624 07:18:51.447734361 9603 httpproxy.cc:54] cannot parse value of 'httpproxy' env var

    ===
    Client Library version:

    google-cloud-pubsub 0.35.4

    6 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  6. Screenshots and Self driven flash based / gif - much helpful for

    Documentation should have screenshots and adobe flash based / gifs which can give visualization to anyone who would like to try / explore.

    6 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  7. Add publisher IP to PubsubMessage

    For improved auditing and security it would be nice if a field were added that specifies the originating IP address of the message.

    5 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  8. gcloud cli does not interoperate with pubsub emulator

    It's not currently possible to work with a local emulator using gcloud. How do I create a topic or subscription without writing code for this purpose?

    5 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  9. Emulator should not need content-type header, or provide proper error response

    The real PubSub API request a publish request (potentially others) without the content-type header.

    The PubSub Emulator returns a 404 error if no content-type header is set, or a wrong content-type header is set.

    404 is the wrong HTTP status code in such a situation (400 is correct), and without a proper error message, this is extremely difficult to debug. I wasted a lot of time debugging whether I set up the topic correctly, and whether I constructed the URL correctly.

    Please either ignore the content-type header (like the PubSub API does), or provide a 400 response with a proper…

    5 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  10. Java process spawned when Pub/Sub emulator runs is not cleaned up

    When you start the Pub/Sub emulator (e.g. gcloud beta emulators pubsub start --project=myprog) there is also a java process which is spawned. I am assuming it contains a jetty process or similar.

    When you stop the Pub/Sub emulator (ctrl + c) this process remains. This causes issues in many ways. Some I have run into:


    1. The port in use is now locked while the java process remains. Starting Pub/Sub emulator again starts it on a new port.

    2. If there was an http push error, and the Pub/Sub emulator is continuously resending the message, it will continue to do so…
    4 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  11. 4 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  12. PROVIDE A SIMPLE NOTIFICATION SYSTEM FOR APP PURCHASE INFO

    I just want to know who bought the app, not fly to mars...
    I just want a simple list of every time a user purchased in my app, or cancelled. That's it. I don't want a whole freaking complicated publication and subscription system. BTW, I've spent the last hour with your documentation, and I am yet to see a single notification.
    Every app developer with in app purchases needs this. None wants to learn your whole cloud system to be able to get this basic information. Thanks

    4 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  13. Pub/Sub message ordering should also support stickiness by key

    Similar to kafka, I expect message ordering key to validate that same subscriber gets all messages tagged by a specific message key.
    If I have to aggregate data by message key, currently I can do it only by a service that cannot be scaled. I would like to be able to scale my service, knowing that messages of the same key will always be received by the same service instance.
    (Tested it by writing messages to a topic, all with the same ordering key. Tried to read from two service instances, using same ordering-enabled subscription. While most of the messages…

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  14. No esta traducida al español.

    No esta traducida al español.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  15. Message groups

    Expose a group ID attribute on the message, and provide the option to block pulling messages with the same group ID until the current message is acknowledged. The pull operation should skip over any messages that are currently locked and instead return the next available message. It's a very useful feature for implementing state machines and other event processing systems.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  16. Enable API key authentication for publisher

    I have a use case where events are published to a pub/sub topic from the client side. As it is client side I cannot authenticate via the methods supported currently for publishing messages. As quite tight restrictions can be placed on what domain is attempting to publish it would be great to open up the possibility of authenticating via an API key

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  17. Provide message publisher over simple socket connection

    I am trying to connect to GCP pubsub from a IBM mainframe. I have a simple socket connection written in C language. I would like to send messages over the socket without the overhead of a HTTP message.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  18. `endsWith` and `contains` should be included here - or pattern matching

    You might all this a feature request but I find it to be defective that you have a function to determine if an attribute "starts with" but not a function for "ends with". This makes it difficult to do things such as pick apart contexts from strings like

    com.mycompany.mydomain.myentity.myevent.v1

    For example, if I wanted to say "subscribe to all messages from mydomain" then hasPrefix(com.mycompany.mydomain) makes sense. But if I wanted to say "subscribe to any domain that publishes myevent" then I would need hasSuffix(myevent.v1).

    That said, I would also need something to determine something in the middle. For example "subscribe…

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  19. Filter with "IN" clause

    Hi,
    Is it possible to add the ability to filter say an attribute that can have a range of values. Instead of saying where attribute:k1 = "value1" AND attribute:k1 = "value2", can we have something like attribute:k1 IN ("value1", "value2")

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  20. Add asyc messages on standard env

    Using the standard environment with automatic scaling (no background threads) there are no or very small chances to buffer messages and send them to pub/sub in a batch rather than one by one (e.g triggered by a client request).

    I would like to see it possible to buffer messages and have the pub sub lib send them either when the buffer is full or after a certain time interval has been reached.

    This would be a very nice performance boost.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  • Don't see your idea?

Cloud Pub/Sub

Categories

Feedback and Knowledge Base