Cloud Pub/Sub

Welcome to the Google Cloud Pub/Sub idea forum. You can submit and vote on ideas here to tell the Pub/Sub team which features you’d like to see.

This forum is for feature suggestions. If you’re looking for help forums, look here:

We can’t wait to hear from you!

  1. Subscriber Message Filtering

    It should be possible for a subscriber to assign a "filter policy" to the topic subscription. E.g. by looking at some header attribute it should be possible to only get a subset of messages.

    398 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    6 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  2. Add Retry Count to Pub/Sub

    The way we plan to work with Pub/Sub is by pulling messages by workers. Let's say certain worker pulls the message but unable to process it. The unacknowledged message goes back to queue and will be pulled by another worker. This loop can continue indefinitely unless the 7 days limit of max message age is reached.

    It would be great to being able to set max retry count which will move the message to dead-messages-queue topic after exhausting the retry limit for the message.

    333 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    6 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  3. Replace returnImmediately with a timeout

    In its current form, projects.subscriptions.pull accepts the following parameters in its request payload: returnImmediately and maxMessages. The former parameter restricts the library authors (including the official GCP driver) into one of two extremes:


    1. Either poll the endpoint periodically (e.g., every 10s) via returnImmediately=true.


    2. Or poll indefinitely via returnImmediately=false.


    The first approach ensures a deterministic message stream with the cost of degraded latency. The second approach does the opposite: a nondeterministic stream with better latency. I think there is a better alternative: introducing a timeout parameter. This will act as knob one can tune to move between these…

    41 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  4. Instance Group auto-scaling based on Cloud Pub/Sub queue count

    It would be great to have an auto-scaling metric for GCE Instance Groups (or k8s 'Jobs') based on a Cloud Pub/Sub's topic count. For asynchronous workloads, as tasks are written into the topic (queue), spin up/down workers from an instance group's template. Additionaly, as Cloud Functions become fully supported, more events will are hoping to move more workloads into a dynamicly scalling into Cloud Pub/Sub - Instance Group workflow.

    31 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    3 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  5. ability to see (read/delete) messages in the console

    It would be great for testing and troubleshooting.

    26 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  6. Remove topic limit

    10,000 topics isn't enough. I need something like 10,000 million.

    25 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  7. 25 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  8. Provide an example of how to work with push subscriptions locally using the emulator

    When working locally I can't seem to get around the https restrictions, IMO a self signed certificate on https://localhost:port/pushendpoint should be sufficient when working locally.

    15 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  9. Allow specific metadata for notifications from Google Cloud Storage

    When GCS is registered as a publisher to a topic, we can specify none as the payload if all we need are the eventType, bucketId, objectId, and objectGeneration fields. This is great since the message data is only roughly 650 bytes for reasonably size bucket names and file names. For most projects, that's all we need. It's great that objectGeneration happens to be included.

    However, if we need just one other field from the object's metadata, say Content-Length for example, which would add roughly 20 more bytes, we're stuck with the entire basket of metadata fields. We'd like to specify…

    14 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  10. A way to specify headers for pubsub push mechanism

    If developers have control over headers the client doesn't have to write anything to add security
    We can just use esp.
    Even though esp looks in to query parameters,
    we have to rotate the token every 24 hours for better security

    12 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  11. 11 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  12. Open up API for domain verification

    There is an existing set of APIs for automating the Search Console piece of getting a webhook url set up for push notifications. However, you still have to manually go into your GCP project to add the webhook to the Verified Domains under credentials in order to use it in your project. It would be great to be able to automate this piece as well, but there isn't an API we can call. Could one be created or opened up?

    8 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  13. 7 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  14. undelivered messages Retains period too low.

    clients cam have vacations up to 6 weeks sometimes. having 7 day retention for those messages too low.

    7 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  15. Please provide a client in C language

    I have lot of stuff already written in C and need a pub/sub client in c to integrate to Google Pub/Sub.

    6 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  16. Add publisher IP to PubsubMessage

    For improved auditing and security it would be nice if a field were added that specifies the originating IP address of the message.

    5 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  17. 4 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  18. Message groups

    Expose a group ID attribute on the message, and provide the option to block pulling messages with the same group ID until the current message is acknowledged. The pull operation should skip over any messages that are currently locked and instead return the next available message. It's a very useful feature for implementing state machines and other event processing systems.

    3 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  19. delayed / scheduled messages

    When publishing a message, accept a DateTime (in the future) which defines when the message will be scheduled. This way a function can be executed in e.g. 30 minutes (with the message content as an input argument).

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  20. gcloud cli does not interoperate with pubsub emulator

    It's not currently possible to work with a local emulator using gcloud. How do I create a topic or subscription without writing code for this purpose?

    2 votes
    Vote
    Sign in
    (thinking…)
    Sign in with: Facebook Google
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1
  • Don't see your idea?

Cloud Pub/Sub

Categories

Feedback and Knowledge Base