Cloud Pub/Sub

Welcome to the Google Cloud Pub/Sub idea forum. You can submit and vote on ideas here to tell the Pub/Sub team which features you’d like to see.

This forum is for feature suggestions. If you’re looking for help forums, look here:

We can’t wait to hear from you!

  1. "Migrate" subscriptions

    If you have to update a subscription's settings (like changing a filter), then the subscription needs to be recreated. The risk here is that, in production, there could be messages pending so the developer has to stop the publisher and allow the subscription to clear before deleting it and recreating it, resulting in some down time.

    It would be helpful to have a feature that could "migrate" one subscription to another. So for example, I would create the new/updated subscription and then ask PubSub to move any unprocessed message from the old one to the new one. There would still…

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  2. `endsWith` and `contains` should be included here - or pattern matching

    You might all this a feature request but I find it to be defective that you have a function to determine if an attribute "starts with" but not a function for "ends with". This makes it difficult to do things such as pick apart contexts from strings like

    com.mycompany.mydomain.myentity.myevent.v1

    For example, if I wanted to say "subscribe to all messages from mydomain" then hasPrefix(com.mycompany.mydomain) makes sense. But if I wanted to say "subscribe to any domain that publishes myevent" then I would need hasSuffix(myevent.v1).

    That said, I would also need something to determine something in the middle. For example "subscribe…

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  3. Dockerized image for emulator

    A docker image for the emulator which doesn’t require the install of the gcloud tools

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  4. Airport Transfer

    AtoB Transfer specializes in a taxi service pickup from and to the airport. A taxi transfer with AtoB is more than just a taxi ride. You are treated like a special guest or a good old friend as our number one priority is total customer satisfaction. We create a unique taxi experience.
    https://atobtransfer.com/

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  5. Provide message publisher over simple socket connection

    I am trying to connect to GCP pubsub from a IBM mainframe. I have a simple socket connection written in C language. I would like to send messages over the socket without the overhead of a HTTP message.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  6. Pub/Sub message ordering should also support stickiness by key

    Similar to kafka, I expect message ordering key to validate that same subscriber gets all messages tagged by a specific message key.
    If I have to aggregate data by message key, currently I can do it only by a service that cannot be scaled. I would like to be able to scale my service, knowing that messages of the same key will always be received by the same service instance.
    (Tested it by writing messages to a topic, all with the same ordering key. Tried to read from two service instances, using same ordering-enabled subscription. While most of the messages…

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  7. Fix the bug in emulator v331 where logging doesn't work

    I'm getting this error when starting the pubsub emulator (version 331):

    [pubsub] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
    [pubsub] SLF4J: Defaulting to no-operation (NOP) logger implementation
    [pubsub] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
    I'm running on Debian testing, with openjdk-11-{jre,jdk}:amd64 version 11.0.11+4-1.

    I tried installing packages: libslf4j-java and liblog4j1.2-java but it did not work.

    I found a hacky workaround, which I have described here on StackOverflow: https://stackoverflow.com/questions/66631222/pubsub-emulator-slf4j-error-on-start-no-logging/66631906

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  8. Spark Structured Streaming

    Apache Spark is the top framework for Big Data processing, widely adopted and with more contributors. Not being able to use Structured Streaming in Pub/Sub is a mayor limitation.

    9 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  9. Java process spawned when Pub/Sub emulator runs is not cleaned up

    When you start the Pub/Sub emulator (e.g. gcloud beta emulators pubsub start --project=myprog) there is also a java process which is spawned. I am assuming it contains a jetty process or similar.

    When you stop the Pub/Sub emulator (ctrl + c) this process remains. This causes issues in many ways. Some I have run into:


    1. The port in use is now locked while the java process remains. Starting Pub/Sub emulator again starts it on a new port.

    2. If there was an http push error, and the Pub/Sub emulator is continuously resending the message, it will continue to do so…
    4 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  10. Subscriber Filtering Messages

    Filter messages based on given substring.
    Eg:- has(attributes.KEY, "SUBSTRING")

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  11. hi. Please provide samples and more info about use cases

    I have a python Django app that use Celery for asynchronous task, and rabbitMQ to send emails. I want to migrate my app to GCP, and I want to have the best architecture, more efficient, best performance, best practices. So, I want a direction in what should I use, or replace. Thank you.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  12. High CPU usage in the emulator

    The pub/sub emulator has a very high CPU usage, both during use but especially when idling. This appears to be more severe the more subscribers are added to a topic.

    The CPU usage is so high that it limits it's use case for local development, at least on weaker machines.

    As a suggestion I would propose to completely rewrite the emulator in Go, but I do understand that might be unfeasible.

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  13. Plusen control fuel saving project

    I'm making project plusen contorl fule saving projact on new power generation project is completed your support depends think

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  14. publicity entertainment Communication

    Nice Communication friendship publicity others

    2 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  15. 1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  16. Demasiada información y muy difícil de encontrar lo que uno busca

    Es muy difícil encontrar lo que uno busca, demasiada información y muchas horas perdidas para finalmente no encontrar lo que mi proyecto necesita. Simplemente queremos cada vez que un email llegue a la cuenta nuestra de gmail, se envié un payload con dicha información a un endpoint en nuestro server. He probado a configurar el push con el endpoint (siguiendo el ejemplo de la página de ayuda, pero en vez de pull seleccionado push), pero el mensaje nunca llega a nuestro servidor.

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  17. Pub/Sub emulator huge delay on pull subscriptions

    I'm experiencing huge delays (which are blocking the development) from time to time on the pubsub emulator pull subscriptions.
    I'm starting the pubsub emulator via docker using latest google/cloud-sdk alpine image (I've tried also debian based image):

    FROM google/cloud-sdk:alpine
    
    RUN gcloud components install --quiet \
    pubsub-emulator \
    beta
    EXPOSE 8538
    VOLUME /opt/data

    Starting the docker with command:

    gcloud beta emulators pubsub start \
    
    --data-dir=/opt/data \
    --host-port=127.0.0.1:8538 \
    --host-port=0.0.0.0:8538 \
    --user-output-enabled \
    --log-http \
    --verbosity=debug

    I create topics and subscriptions via curl, I'm trying to create pull message subscriptions the same way.
    When I create pull message, there is sometimes 5…

    27 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    1 comment  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
  18. Filter with "IN" clause

    Hi,
    Is it possible to add the ability to filter say an attribute that can have a range of values. Instead of saying where attribute:k1 = "value1" AND attribute:k1 = "value2", can we have something like attribute:k1 IN ("value1", "value2")

    3 votes
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  19. How to use the pubsub emulator in multiple languages

    There is one brief example in python. Could other examples be created - like node?

    1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Feature Request  ·  Flag idea as inappropriate…  ·  Admin →
  20. 1 vote
    Vote

    We're glad you're here

    Please sign in to leave feedback

    Signed in as (Sign out)
    You have left! (?) (thinking…)
    0 comments  ·  Defect  ·  Flag idea as inappropriate…  ·  Admin →
← Previous 1 3 4 5
  • Don't see your idea?

Cloud Pub/Sub

Categories

Feedback and Knowledge Base