BUG: missing candles for ONE_MINUTE, FIVE_MINUTE, and ONE_DAY

[michael.whittle]

I’ve discovered an irritating bug with the candles endpoint.

It’s now mandatory to specify the start and end unix timestamp. With Coinbase Pro if you didn’t specify it you just got the last 300 entries. There is validation on the endpoint now that prevents you from specifying a time frame larger than 300 x granularity.

The ONE_MINUTE, FIVE_MINUTE, and ONE_DAY are missing entries.

For example, right now for BTC-GBP you can only get the last 128 minutes and 256 five minutes. You can’t increase the start and end to make sure you get the max 300 entries. It will throw an error to say the time frame is too long even though you are under the 300 limit.

“start and end argument is invalid - number of candles requested should be less than 300”

For the ONE_DAY it’s slightly different because it won’t give you 300 days. It gives you 299 because you are on the current day.

It should give you 300 entries regardless. This wasn’t a great change from Coinbase Pro. Those start and end should not be mandatory and the validation should be on getting 300 entries back, not based on the timeframe.

If the devs are reading these forums… please fix this.

[bazinga]

Hi @michael.whittle! We appreciate your interest in using Coinbase Cloud products and services, and we’re glad to help! But in order for us to accurately address your concerns, we would like you to us first with the following information:

• Can you elaborate further when you said “For example, right now for BTC-GBP you can only get the last 128 minutes and 256 five minutes. You can’t increase the start and end to make sure you get the max 300 entries. It will throw an error to say the time frame is too long even though you are under the 300 limit.”?
• If possible, can you share with us the timestamp values you used for the start and end parameters that resulted in these data set values you were citing for the pair BTC-GBP?
• Can you also share with us the values you tried to utilize for the start and end parameters but resulted in an error that says “start and end argument is invalid - number of candles requested should be less than 300"?
• If you have any additional information that can demonstrate your concern, feel free to share it with us. Please ensure to omit any sensitive information that may compromise your account.

For your other concern about having the /candles endpoint of Advanced Trade similar to that of Coinbase Pro, wherein start and end parameters are not required and the endpoint would by default return 300 candles on a single request, please note that we have logged this as feedback for now with our internal teams so we can continue improving our user experience. Most new features and improvements to our products come directly from feedback like yours, so it’s very valuable to us. While we can’t offer any specific timeline for adding features and improvements, we are constantly working to build products our customers will love.If you want to stay up to date on the latest from Coinbase Cloud, you can also bookmark the following webpage and subscribe to email updates at the bottom of the page: https://www.coinbase.com/cloud/discover

We hope this helps. We will await the above details we have requested in order for us to further investigate your concern. Once received, we’ll work quickly to respond with more context and accurately address your concerns.

Thank you and have a great day!

[michael.whittle]

Thanks for the reply.

I can give you one example here for BTC-GBP.

{‘start’: 1679812045, ‘end’: 1679830046, ‘granularity’: ‘ONE_MINUTE’}

1679812045 = Sun Mar 26 2023 06:27:25 GMT+0000
1679830046 = Sun Mar 26 2023 11:27:26 GMT+0000

This should be exactly 300 hours, calculated like this:

        if granularity == 60:
            payload = {
                "start": int(time.mktime((datetime.now() - timedelta(minutes=300)).timetuple())),
                "end": int(time.time()),
                "granularity": "ONE_MINUTE",
            }

URL:
https://api.coinbase.com/api/v3/brokerage/products/BTC-GBP/candles?start=1679812045&end=1679830046&granularity=ONE_MINUTE’

I convert the JSON response into a Pandas dataframe.

As you can see 84 minutes returned. Even with a basic visual inspection you can see that 11:19, 11:20, 11:22, , 11:24, and 11:26 are missing.

If I increase the start and end date even by 1 minute I get the error message to say that it exceeds 300. Yes, 300 minutes would be returned if all the data is there but with the missing entries it’s very short.

It’s the same for the FIVE_MINUTE as well. Missing entries.

ONE_DAY returns only 299 days because it doesn’t count the current day and doesn’t allow you to retrieve more than 300 days. If 301 day timeframe was allowed then you would get the full 300 days.

This should definitely mirror what Coinbase Pro has to retrieve a max of 300 records, not done how it is done currently. The “start” and “end” should not be mandatory as well.

Just a few more points to add to your list…

In Coinbase Pro you are using ISO8601 time formats but in Advanced Trade you are using UNIX timestamps. Either would be fine but if you want to make a migration between the two easy it would have been better to be consistent.

You are also missing a time endpoint. All exchanges have it including Coinbase Pro. My app also calls the Coinbase Pro time API to confirm the current time of the exchange. This is not in the mapping list in the documentation. All exchanges have this endpoint so it should be included as well.

[HakunaMatata]

@michael.whittle Thank you for the detailed explanation. We will be relaying this to our internal teams and we will update you regarding this concern as soon as we hear from them.

We appreciate your patience and understanding.

[jmicko]

I’ve been frustrated with this myself for some time. Kinda looks like there just isn’t data for some time periods. If you look at the charts on the main Coinbase website, you can see that some sections of time are just missing. Maybe no one traded anything for entire minutes at a time? But then I would still want to see an entry returned with null or 0 values so that I at least know there’s supposed to be a gap in the data. As it is right now, there’s just no way to know if it’s a bug or intentional. It also seems kinda wild that no one would be trading BTC-GBP for entire days at a time.

[Lioness]

Hi @jmicko! For the details regarding your inquiry, we will be relaying this to our internal teams as well. We’ll get back to you once we have more information. Keep in touch!

[michael.whittle]

I don’t think there is a problem as such for the ONE_DAY. I just don’t think it will give you more than 299 days as the current day would not have completed and the date range validation won’t allow you to be larger than 300 days.

There is definitely a problem with ONE_MINUTE and FIVE_MINUTE. It’s highly unlikely that no trades were done at all for BTC-GBP. The problem with creating an empty candle is it will break moving averages and technical analysis. I don’t really mind the gap (if it’s a real gap), but I want to see 300 data points including the gaps.

[michael.whittle]

@bazinga @Lioness, what ever happened with this? It’s been months and still looks like the start and end UNIX timestamp is still mandatory. The Coinbase Advanced Trade API is still unusable for me in its current stage, I’m still using Coinbase Pro. I have users in my PyCryptoBot project complaining about issues with the new API. This really needs to be fixed to be able to use it properly.

You should do what all other exchanges do, including Coinbase Pro.

1. Not specifying a start and end UNIX timestamp - retrieve the latest 300 (for Coinbase Pro) or 500 (for Binance) candles (by default).

2. Only specify a start UNIX timestamp - retrieve the first 300 candles from the start UNIX timestamp.

3. Only specify a end UNIX timestamp - retrieve the last 300 candles from the end UNIX timestamp.

The way it is now making it mandatory to specify the start and end UNIX timestamp and still having a limit on how many candles can be retrieved doesn’t make sense. What is worse is even if you do calculate the 300 candle range manually, you sometimes have missing candles which means it still not work properly or be consistant.

It really has been months now and no resolution to this. Please can you escalate this to be fixed.

[ATuser123]

for what its worth, looks like they released a channel for candles this week: WS Channels | Coinbase Cloud

[michael.whittle]

The API is still really broken. I try and retrieve ADA-GBP 15 minute candles using the maximum start and end it will allow and I only get 151 candles back. Coinbase Pro it’s 300! This API is so broken it’s unreal and it’s been months and they can’t seem to make to fix something basic. I see they aren’t allowing anyone new on Coinbase Pro, or at least you can’t create new API keys. So they are going to force people to use people onto this terrible unfinished API with no support.

[tijn]

I’m also frustrated by this. When I request candlestick data between Jan 1st 2018 and March 4th 2024, only the 1 day and 6 hour granularities are complete. All the shorter ones have loads of gaps. As mentioned this makes it impossible to do meaningful analysis on the data. I was looking forward to spending my money (and hopefully make some) on your platform but I think I’ll have to find another one that offers a working API.

[michael.whittle]

They should never have sunsetted Coinbase Pro before Advanced Trade was ready. What a disaster. The API and documentation for it is really poor. It feels like a hobby project when working with it.