tag:status.getdbt.com,2005:/historydbt Cloud Status - Incident History2024-12-12T16:26:51-05:00dbt Cloudtag:status.getdbt.com,2005:Incident/231750702024-12-12T16:26:51-05:002024-12-12T16:26:51-05:00Small amount of users on Teams plan experiencing issues with Billing page in dbt Cloud UI<p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>16:26</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>15:49</var> EST</small><br><strong>Monitoring</strong> - We believe the issue has been resolved and we're monitoring the fix.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>14:32</var> EST</small><br><strong>Update</strong> - We're deploying a fix now.</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>13:17</var> EST</small><br><strong>Identified</strong> - We have identified an issue with the Billing page in the dbt Cloud UI that affected a small amount of Teams customers. A fix is being implemented, and we will provide an update shortly.</p>tag:status.getdbt.com,2005:Incident/231628882024-12-12T01:28:58-05:002024-12-12T01:28:58-05:00dbt deps failing in IDE when using ADO integration in Beta<p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>01:28</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved. We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p><p><small>Dec <var data-var='date'>11</var>, <var data-var='time'>21:29</var> EST</small><br><strong>Monitoring</strong> - We have deployed a fix for the reported issue and are monitoring this closely. Please contact us at [email protected] with any questions or concerns.</p><p><small>Dec <var data-var='date'>11</var>, <var data-var='time'>16:11</var> EST</small><br><strong>Identified</strong> - We have identified an issue with the new Private Packages beta feature for Azure DevOps that resulted in dbt deps failures in the IDE if referencing a package with the new private syntax. A fix has been implemented, and will roll out shortly.</p>tag:status.getdbt.com,2005:Incident/231182492024-12-09T01:15:34-05:002024-12-09T01:15:34-05:00Error when previewing results in the IDE<p><small>Dec <var data-var='date'> 9</var>, <var data-var='time'>01:15</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally.<br /><br />Please contact Support via email at [email protected] if you continue to experience delays and are unsure of the root cause.<br /><br />We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p><p><small>Dec <var data-var='date'> 9</var>, <var data-var='time'>00:10</var> EST</small><br><strong>Monitoring</strong> - We have deployed a fix for the reported issue. Previewing results in the IDE has returned to its normal state and we are continuing to monitor.</p><p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>21:59</var> EST</small><br><strong>Identified</strong> - We have identified the root cause of the reported issue when previewing results in the IDE. A fix is being implemented, and we will provide an update shortly. Thanks for your patience.</p><p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>20:59</var> EST</small><br><strong>Investigating</strong> - We're currently investigating an issue with users being unable to preview results in the IDE and seeing an error "Something went wrong. Please try reloading the page. If the problem persists, contact our support team for help".<br /><br />We will continue sharing updates as we progress. Please contact us at [email protected] with any questions or concerns. Your patience and understanding are greatly appreciated.</p>tag:status.getdbt.com,2005:Incident/230699232024-12-04T14:46:30-05:002024-12-04T14:46:30-05:00Increased http 500 errors in dbt Copilot<p><small>Dec <var data-var='date'> 4</var>, <var data-var='time'>14:46</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 1:00 PM EST.<br /><br />Between the hours of 7:30 AM EST and 1 pm EST, there were increased errors being reported from dbt Copilot within the dbt Cloud IDE. Users may have noticed errors with being unable to complete inline generation or semantic model generation. This was due to a breaking change that has since been reverted, and we have verified there are no further errors occurring upon monitoring after fix. <br />Please contact Support via chat or email [email protected] if you continue to experience delays and are unsure of the root cause.<br /><br />We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p><p><small>Dec <var data-var='date'> 4</var>, <var data-var='time'>13:48</var> EST</small><br><strong>Monitoring</strong> - We have deployed a fix for the impacted API service that was causing 500 errors to occur in dbt Copilot. dbt Copilot has returned to its normal state and we are continuing to monitor.</p>tag:status.getdbt.com,2005:Incident/229783662024-11-27T02:41:58-05:002024-11-27T02:41:58-05:00Increase in errors being returned on API calls<p><small>Nov <var data-var='date'>27</var>, <var data-var='time'>02:41</var> EST</small><br><strong>Resolved</strong> - This issue has been resolved. Please contact Support via email at [email protected] if you continue to experience issues and are unsure of the root cause.</p><p><small>Nov <var data-var='date'>27</var>, <var data-var='time'>01:01</var> EST</small><br><strong>Monitoring</strong> - We have deployed a fix for the dbt Cloud Administrative API that was causing an increased rate in 400 errors being returned in responses. The dbt Cloud Administrative API has returned to its normal state and we are continuing to monitor.</p><p><small>Nov <var data-var='date'>27</var>, <var data-var='time'>00:15</var> EST</small><br><strong>Identified</strong> - We have identified an issue with the dbt Cloud Administrative API that resulted in the increase of status 400 errors being returned in responses. A fix is being implemented, and we will provide an update shortly.</p><p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>15:50</var> EST</small><br><strong>Investigating</strong> - We're investigating an issue with the dbt Cloud Administrative API that is causing an increased rate in 400 errors being returned in responses. The team is working on a resolution and we will provide updates as soon as new information becomes available.</p>tag:status.getdbt.com,2005:Incident/229774502024-11-26T14:28:44-05:002024-11-26T14:28:44-05:00Semantic Layer queries failing for a small percentage of customers<p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>14:28</var> EST</small><br><strong>Resolved</strong> - We have identified a fix and rolled it out. We have monitored the status and are no longer seeing any failures with the queries. Please feel free to reach out to our Support team at [email protected] in case you continue to experience these errors.</p><p><small>Nov <var data-var='date'>26</var>, <var data-var='time'>14:03</var> EST</small><br><strong>Identified</strong> - We received internal alerts that Semantic Layer queries are taking a long time to run for a small percentage of customers. Affected customers would expect to see an “Internal error with code [70635]” error message.<br />We have identified the cause and are currently working on a fix.</p>tag:status.getdbt.com,2005:Incident/229311732024-11-22T01:57:22-05:002024-12-04T00:16:41-05:00Investigating dbt Cloud deployment runs failing due to PrivateLink errors<p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>01:57</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally.<br /><br />Please contact Support via email [email protected] if you continue to experience delays and are unsure of the root cause.<br /><br />We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>01:22</var> EST</small><br><strong>Monitoring</strong> - Confirming that All instances are now operational. We are continually working on returning US Cell1 to an operational state as well. Thank you for your patience.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>01:17</var> EST</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>01:02</var> EST</small><br><strong>Identified</strong> - We have identified the underlying issue and a fix is being implemented, and we will provide an update shortly. Thanks for your patience.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>00:05</var> EST</small><br><strong>Investigating</strong> - We're investigating an issue with dbt Cloud deployment runs failing due to error related to PrivateLink connection. The team is working on a resolution and we will provide updates at approximately 30 minute intervals or as soon as new information becomes available.</p>tag:status.getdbt.com,2005:Incident/229147172024-11-20T15:32:06-05:002024-11-20T15:32:06-05:00dbt Cloud job runs being cancelled<p><small>Nov <var data-var='date'>20</var>, <var data-var='time'>15:32</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 3:30 PM Eastern. <br /><br />Between 1:16 PM Eastern and 1:58 PM Eastern an issue with Job scheduler resulted in Runs being cancelled with a message of "This run timed out after 30 minutes of inactivity. Please retry the run, or contact support if this job continues to fail." We have put measures in place to prevent a recurrence and Jobs have returned to normal. <br /><br />Please contact Support via email at [email protected] if you continue to experience cancellations and are unsure of the root cause.</p><p><small>Nov <var data-var='date'>20</var>, <var data-var='time'>14:53</var> EST</small><br><strong>Monitoring</strong> - We have deployed a fix for the Job scheduler that resulted in Runs being cancelled with a message of "This run timed out after 30 minutes of inactivity. Please retry the run, or contact support if this job continues to fail." Jobs have returned to their normal state and we are continuing to monitor.</p><p><small>Nov <var data-var='date'>20</var>, <var data-var='time'>14:37</var> EST</small><br><strong>Identified</strong> - We have identified an issue with the Job scheduler that resulted in Runs being cancelled with a message of "This run timed out after 30 minutes of inactivity. Please retry the run, or contact support if this job continues to fail." This impacted US Multi-Tenant instances. A fix has been implemented, and we will provide an update shortly.</p>tag:status.getdbt.com,2005:Incident/228933222024-11-18T20:41:43-05:002024-11-18T20:41:43-05:00North America Cell 2 experiencing delays in metadata ingestion<p><small>Nov <var data-var='date'>18</var>, <var data-var='time'>20:41</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally.<br /><br />Please contact Support via email [email protected] if you continue to experience delays and are unsure of the root cause.<br /><br />We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p><p><small>Nov <var data-var='date'>18</var>, <var data-var='time'>20:26</var> EST</small><br><strong>Monitoring</strong> - We have deployed a fix for the identified latency with metadata ingestion and are currently monitoring the performance. Please reach out to [email protected] with any questions or concerns.</p><p><small>Nov <var data-var='date'>18</var>, <var data-var='time'>19:42</var> EST</small><br><strong>Identified</strong> - We have identified a delay with metadata ingestion which is causing stale data (by a few minutes) when viewing in dbt Cloud Explorer or when querying the API. A fix is being implemented, and we will provide an update shortly. Thanks for your patience.</p>tag:status.getdbt.com,2005:Incident/228084032024-11-08T20:31:20-05:002024-11-08T20:31:20-05:00Semantic Layer errors for BigQuery customers<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>20:31</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 2024-11-09 01:26 UTC. Please contact Support via email [email protected] if you continue to experience issues and are unsure of the root cause.</p><p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>19:19</var> EST</small><br><strong>Investigating</strong> - We're investigating an issue with the Semantic Layer that is causing errors for some customers on BigQuery. This is impacting Semantic Layer queries anytime after 2024-11-08 15:17 UTC. While we work on a resolution, our engineers have rolled back to a previous state. We will provide more updates as they become available.</p>tag:status.getdbt.com,2005:Incident/227706302024-11-05T07:23:44-05:002024-11-05T07:23:44-05:00Explorer Opening to Blank Page<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>07:23</var> EST</small><br><strong>Resolved</strong> - The issue has been resolved, and Explorer is now functioning normally as of 7:15 Eastern. <br />Between 20:58 Eastern 11/04/2024 and 7:15 Eastern 11/05/2024 an issue with Explorer resulted in a blank page for users using Project Permissions to grant Metadata read access. We have put measures in place to prevent a recurrence and Explorer has returned to normal. <br /><br />Please contact Support via email [email protected] if you continue to experience issues and are unsure of the root cause.</p><p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>06:56</var> EST</small><br><strong>Monitoring</strong> - We have deployed a fix for Explorer that was causing a blank page for users using Project Permissions to grant Metadata read access. Explorer should be returned to its normal state and we are continuing to monitor.</p><p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>06:23</var> EST</small><br><strong>Identified</strong> - We have identified an issue with Explorer that resulted in a blank page. This is happening only to users using Project Permissions to grant Metadata read access. A fix is being implemented, and we will provide an update shortly.</p><p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>06:01</var> EST</small><br><strong>Investigating</strong> - We're investigating an issue with Explorer that is causing a blank page to be seen when opened. This is impacting a subset of accounts, in all regions anytime after 8:58 PM EST 11/04/2024. The team is working on a resolution and we will provide more updates as they become available.</p>tag:status.getdbt.com,2005:Incident/227661292024-11-04T19:02:28-05:002024-11-04T19:02:28-05:00Scheduled jobs delayed in US Multi Tenant<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>19:02</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>19:01</var> EST</small><br><strong>Identified</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 4 PM EST. <br /><br />Between 3:30 PM EST and 4:00 PM EST, a planned change in the architecture of the dbt Cloud job scheduler lead to performance degradation causing delays in scheduled jobs. <br /><br />Please contact Support via chat or email [email protected] if you experience any delays and are unsure of the root cause.<br /><br />We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p>tag:status.getdbt.com,2005:Incident/227275152024-11-01T11:50:09-04:002024-11-01T11:50:09-04:00Users reporting failing production jobs when using Databricks adapter<p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>11:50</var> EDT</small><br><strong>Resolved</strong> - This issue has been resolved, and all affected systems are now functioning normally as of 11:48 AM EDT. Please contact us via [email protected] if you experience any issues with runs while using the Databricks adapter.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>18:00</var> EDT</small><br><strong>Update</strong> - We are continuing to work on the fix for this issue, and it should be deployed within the next 24 hours. Please contact us on [email protected] if you experience any issues with runs while using the Databricks adapter.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>15:28</var> EDT</small><br><strong>Identified</strong> - We have identified an issue with incremental models within scheduled jobs that resulted in failed runs while using Databricks on Versionless. A fix is being implemented, and we will provide an update shortly.</p>tag:status.getdbt.com,2005:Incident/227286462024-10-31T18:13:09-04:002024-10-31T18:13:09-04:00Small number of job runs failing to cancel<p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>18:13</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 21:51 UTC.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>17:51</var> EDT</small><br><strong>Monitoring</strong> - We have deployed a fix to the job scheduler that was preventing job runs from being canceled. The scheduler has returned to its normal state and we are continuing to monitor.</p><p><small>Oct <var data-var='date'>31</var>, <var data-var='time'>17:50</var> EDT</small><br><strong>Investigating</strong> - We're investigating an issue with the job scheduler that is preventing runs from being canceled. This is impacting runs in all of dbt Cloud's multi-tenant environments. The team is working on a resolution and we will provide updates as soon as new information becomes available.</p>tag:status.getdbt.com,2005:Incident/227179712024-10-30T19:27:58-04:002024-10-30T19:27:58-04:00Delays in job runs on US Multi-tenant instance<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>19:27</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 23:08 UTC.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>18:58</var> EDT</small><br><strong>Monitoring</strong> - We have deployed a fix to our run pods that was causing delayed job runs. Run pods have returned to their normal state and we are continuing to monitor.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>18:54</var> EDT</small><br><strong>Investigating</strong> - We're investigating an issue that is causing delays in job runs. This is currently impacting dbt Cloud's US Multi-tenant instance. The team is working on a resolution and we will provide updates as soon as new information becomes available.</p>tag:status.getdbt.com,2005:Incident/227153512024-10-30T17:40:00-04:002024-10-30T17:40:00-04:00Delay in triggering of scheduled runs<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>17:40</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 20:20 UTC. Please contact Support via email [email protected] if you continue to experience delays and are unsure of the root cause.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>14:35</var> EDT</small><br><strong>Monitoring</strong> - We have deployed a fix for the dbt Cloud job scheduler that was causing scheduling delays. The job scheduler has returned to its normal state and we are continuing to monitor.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>14:08</var> EDT</small><br><strong>Identified</strong> - We have identified an issue with scheduled runs that resulted in delays in runs being triggered. A fix is being implemented, and we will provide an update shortly.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>13:19</var> EDT</small><br><strong>Investigating</strong> - We're investigating an issue with the dbt Cloud job scheduler that is causing slight delays in triggering runs. This is impacting scheduled runs in our US Multi-Tenant instance anytime after 16:57 UTC. The team is working on a resolution and we will provide updates as soon as new information becomes available.</p>tag:status.getdbt.com,2005:Incident/226239952024-10-21T19:38:20-04:002024-10-21T19:38:20-04:00dbt Cloud CLI commands failing<p><small>Oct <var data-var='date'>21</var>, <var data-var='time'>19:38</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 7:38 PM Eastern. <br /><br />Between 9:56 AM Eastern and 6:30 PM Eastern an issue with dbt Cloud CLI resulted in a "Field 'results' of type Sequence[RunResultOutput] in RunResultsArtifact has invalid value" error when calling dbt commands for a deferred Job. We have put measures in place to prevent a recurrence and dbt Cloud CLI has returned to normal. <br /><br />Please contact Support via email at [email protected] if you continue to experience error and are unsure of the root cause.</p><p><small>Oct <var data-var='date'>21</var>, <var data-var='time'>18:59</var> EDT</small><br><strong>Monitoring</strong> - We have deployed a fix for the dbt Cloud CLI that was causing a "Field 'results' of type Sequence[RunResultOutput] in RunResultsArtifact has invalid value" error when calling dbt commands for a deferred Job. The dbt Cloud CLI has returned to its normal state and we are continuing to monitor.</p><p><small>Oct <var data-var='date'>21</var>, <var data-var='time'>18:17</var> EDT</small><br><strong>Identified</strong> - We have identified an issue with dbt Cloud CLI that is causing a "Field 'results' of type Sequence[RunResultOutput] in RunResultsArtifact has invalid value" error when calling dbt commands for a deferred Job. This is impacting dbt Cloud CLI in all regions and instances anytime after 9:56 AM Eastern. A fix is being implemented, and we will provide an update shortly.</p>tag:status.getdbt.com,2005:Incident/226215342024-10-21T14:16:42-04:002024-10-21T14:16:42-04:00Increased error rate in Cloud Artifacts API<p><small>Oct <var data-var='date'>21</var>, <var data-var='time'>14:16</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 1:00 PM Eastern.<br /> <br />Between 12:35 PM Eastern and 12:55 PM Eastern an issue with an internal API resulted in issues related to cross project references, auto exposures, and accessing account insights. We have put measures in place to prevent a recurrence and these systems have returned to normal. <br /><br />Please contact Support via email at [email protected] if you continue to experience issues and are unsure of the root cause.</p><p><small>Oct <var data-var='date'>21</var>, <var data-var='time'>13:20</var> EDT</small><br><strong>Monitoring</strong> - There was a breaking change made to an internal API within US Multi Tenant and US Multi Cell that impacted dbt Cloud's ability to resolve cross project references, auto exposures, and accessing account insights. This breaking change has been rolled back and we are monitoring. We will provide another update within 30 minutes.</p>tag:status.getdbt.com,2005:Incident/225981562024-10-19T00:24:58-04:002024-10-19T00:24:58-04:00Post Infrastructure Upgrade for North America Cell 3 degraded performance<p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>00:24</var> EDT</small><br><strong>Resolved</strong> - This issue has been resolved.</p><p><small>Oct <var data-var='date'>19</var>, <var data-var='time'>00:19</var> EDT</small><br><strong>Identified</strong> - Upon completion of the infrastructure upgrade for North America Cell 3, we experienced 2 minutes of unavailability which resulted in a small percentage of production job runs to fail. This has since been resolved, please contact [email protected] with any questions or concerns.</p>tag:status.getdbt.com,2005:Incident/225933312024-10-18T23:00:02-04:002024-10-19T00:10:15-04:00Infrastructure Upgrade for North America Cell 3<p><small>Oct <var data-var='date'>18</var>, <var data-var='time'>23:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>18</var>, <var data-var='time'>22:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>18</var>, <var data-var='time'>13:21</var> EDT</small><br><strong>Scheduled</strong> - We will be performing upgrades on our infrastructure. We expect to see a small amount of unavailability (less than 5 min) impacting dbt Cloud only in US Cell 3 at some point in the maintenance window period.</p>tag:status.getdbt.com,2005:Incident/225847572024-10-17T22:26:45-04:002024-10-17T22:26:45-04:00Newly migrated accounts from US Multitenant instance to Multicell3 instance unable to access account<p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>22:26</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally.<br /><br />Please contact Support via email [email protected] with any issues or concerns.<br /><br />We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>21:53</var> EDT</small><br><strong>Monitoring</strong> - We have deployed a fix for the reported issue. dbt Cloud access on Multicell3 has returned to its normal state and we are continuing to monitor.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>21:28</var> EDT</small><br><strong>Identified</strong> - We have identified an issue with newly migrated accounts from US Multitenant instance to Multicell3 instance unable to access account. The team are currently implementing a fix for this issue. Please contact us at [email protected] with any questions or concerns.</p>tag:status.getdbt.com,2005:Incident/225833192024-10-17T19:57:08-04:002024-10-17T19:57:08-04:00A small percentage of dbt Cloud Production job run statuses not updating and getting cancelled (US Multitenant)<p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>19:57</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally.<br /><br />Please contact Support via email [email protected] if you continue to experience delays and are unsure of the root cause.<br /><br />We understand how critical dbt Cloud is to your ability to get work done day-to-day and your experience matters to us. We’re grateful to you for your patience during this incident.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>19:32</var> EDT</small><br><strong>Monitoring</strong> - We have deployed a fix for the reported issue. Scheduled production jobs have returned to the normal state and we are continuing to monitor. Please contact us on [email protected] with any questions or concerns.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>19:04</var> EDT</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>18:55</var> EDT</small><br><strong>Identified</strong> - We have identified an issue with a small percentage of dbt Cloud production job runs in US Multitenant instance not updating the status, which is resulting in the runs timing out and cancelling in 30 minutes. We are currently working on implementing a fix for this issue.</p>tag:status.getdbt.com,2005:Incident/225568782024-10-15T14:46:30-04:002024-10-15T14:46:30-04:00US Multi-Cell 1 UI is inaccessible<p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>14:46</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 2:44pm ET. We have put measures in place to prevent a recurrence and access to the UI in US Multi-Cell 1 been restored. Please contact Support via chat or email [email protected] if you continue to experience delays and are unsure of the root cause.</p><p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>14:40</var> EDT</small><br><strong>Identified</strong> - We have identified an issue with US Multi-Cell 1 that resulted in the site being inaccessible. This is currently only affecting the UI. Scheduled jobs, adhoc jobs, and API calls are not affected. A fix is being implemented, and we will provide an update shortly.</p>tag:status.getdbt.com,2005:Incident/223664412024-10-02T16:54:43-04:002024-10-02T16:54:43-04:00"Can not serialize 'Undefined' object" error returned on dbt invocation in IDE and scheduled runs<p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>16:54</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 2:00 PM Eastern. <br /><br />Between 4:30 PM Eastern on October 1st, 2024 and 2:00 PM Eastern October 2nd, 2024 an issue with the IDE and scheduled runs resulted in "Can not serialize 'Undefined' object" errors on invocation. <br /><br />Please contact Support via email at [email protected] if you continue to experience errors and are unsure of the root cause.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>12:57</var> EDT</small><br><strong>Monitoring</strong> - We have deployed a fix for the IDE and scheduled runs that was resulting in "Can not serialize 'Undefined' object" errors on invocation. The IDE and scheduled runs have returned to their normal state and we are continuing to monitor.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:47</var> EDT</small><br><strong>Identified</strong> - We have identified the issue that resulted in dbt returning "Can not serialize 'Undefined' object" errors on invocation. A fix is being implemented, and we will provide an update shortly.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:43</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:36</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:28</var> EDT</small><br><strong>Investigating</strong> - We are investigating an issue that's causing the error message "Can not serialize 'Undefined' object" to be returned on dbt invocations in both the IDE and scheduled runs. The team is working on a resolution and we will provide updates at approximately 15 minute intervals or as soon as new information becomes available.</p>tag:status.getdbt.com,2005:Incident/223431692024-10-01T02:56:16-04:002024-10-01T02:57:37-04:00'no logs found' occurring on dbt Cloud jobs in MC US1 instance only<p><small>Oct <var data-var='date'> 1</var>, <var data-var='time'>02:56</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved, and all affected systems are now functioning normally as of 01:00 UTC.<br /><br />Please contact dbt Labs Support team via the UI chatbot or [email protected] if you have any questions.</p><p><small>Sep <var data-var='date'>30</var>, <var data-var='time'>21:09</var> EDT</small><br><strong>Monitoring</strong> - We've deployed a fix to the impacted instance and we are continuing to monitor. Jobs starting from 01:00 UTC onwards are expected to complete with their logs. Please contact dbt Labs Support team via the UI chatbot or [email protected] if you have any questions or seeing the issue persisting with new jobs.</p><p><small>Sep <var data-var='date'>30</var>, <var data-var='time'>19:40</var> EDT</small><br><strong>Update</strong> - We are currently rolling out a fix to mitigate the issue with ‘no logs found’ occurring on dbt Cloud jobs.<br /><br />Please note that there may still be a few impacted jobs that are running on old nodes as the fix is being rolled out. Please reach out to Support at [email protected] if you have any questions.</p><p><small>Sep <var data-var='date'>30</var>, <var data-var='time'>18:27</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate the issue with some dbt Cloud jobs in US C1 instances being impacted with no logs found and will provide another update in approximately 30 minutes or as soon as more information becomes available.</p><p><small>Sep <var data-var='date'>30</var>, <var data-var='time'>17:32</var> EDT</small><br><strong>Investigating</strong> - We're investigating an issue with dbt Cloud deploy jobs that is causing no logs found in runs. This is impacting us.c1 Multi Cell instance. The team is working on a resolution and we will provide updates at approximately 30 minute intervals or as soon as new information becomes available.</p>