Troubleshooting Workflows on CloudFerro Cloud

A problem may affect access to the service, the instance itself, or a task running inside it. An instance can be Running while an individual task has failed; the instance status and the task log describe different parts of the service.

Service and web UI access

My Workflows does not open in My Portal

A fresh sign-in is a useful first check when My Workflows will not open. Clearing the browser cache or trying a private window can also help. If access is still denied, your project administrator or support can check service access. The portal route is illustrated in Access MLOps Studio, AI Experiments, and Workflows from My Portal on CloudFerro Cloud.

Airflow UI is disabled

The Airflow UI button is available only when the instance is Running. Other states are explained in Workflows instances on CloudFerro Cloud.

The Workflow web UI denies access

The email used to sign in must appear in the instance’s member list with a role that grants access. The person managing the instance can check this in My Portal. Platform sign-in and instance membership are separate checks. The permissions for each role are described in Members and roles on CloudFerro Cloud.

Sign-in error messages

User not found or not authorized. Please contact the administrator.

Your sign-in worked, but the instance did not authorize your email. Have the instance manager check and save your membership and role.

Error occurred when signing in. Please try again later.

The instance could not verify access with the platform. Wait a few minutes and sign in again. If colleagues can sign in but the error persists for you, ask your project administrator to check the platform IAM role. It must allow the action mlops:GetAirflowRole, as WorkflowsViewer or WorkflowsAdmin does. Sign out and sign in again after a role change. A temporary platform authorization-service outage can also cause this message.

Internal error occurred when signing in. Please try again later.

An unexpected error interrupted sign-in. Contact support with the instance name and the time of the attempt.

Instance creation problems

Create remains disabled

The Create button remains unavailable while required fields are missing or the storage connection has not succeeded. Instance name is one of those fields, and storage must show CONNECTED. A custom pool with autoscaling also needs a maximum node count greater than its pool size.

Object storage connection fails

The connection uses the S3 endpoint and region, the bucket name, and its access-key pair. A Horizon or main website address will not work as the S3 endpoint. The access check in Object storage for Workflows on CloudFerro Cloud establishes whether the same settings can read and write files outside Workflows.

Git connection fails or gives no visible feedback

The Git check depends on the repository URL, branch, and a token with read permission. Expiration and organization approval can affect that permission. GitHub access uses a token rather than an account password.

The check can finish without a visible success badge. If Create remains disabled, other required fields or the storage check may still be incomplete. Once the instance is running, the appearance of the expected DAG confirms that its file has been obtained and loaded.

Creating lasts longer than 30 minutes

Provisioning normally takes a few minutes. Support can investigate a longer delay with the instance name and the approximate time creation began.

Missing DAGs and outdated code

A new DAG does not appear, or old code remains visible

The instance reads one repository branch, so files saved elsewhere will not appear. A new commit also needs time to be copied and parsed. The DAGs page reports import errors, while an expired repository token can leave old DAGs visible without fetching newer files. The Changes to DAG files section in DAGs and a Git repository on CloudFerro Cloud explains synchronization and token expiry.

The DAG has an import error

The error details identify the Python file that could not be loaded. A correction has to be saved to the configured repository branch before the instance can obtain it. Imports also need to match the instance’s engine version. One DAG’s syntax error does not stop other DAGs from loading.

No module named … refers to one of your files

The import path has to match the location of the helper file. The Folders and helper files section in DAGs and a Git repository on CloudFerro Cloud shows the supported layouts.

A helper file is missing at runtime

The task’s file path needs to match the helper’s location. The Folders and helper files section in DAGs and a Git repository on CloudFerro Cloud lists the supported file types. Shell scripts are invoked with bash script.sh because synchronization does not preserve executable permissions.

Failed or queued tasks and unexpected results

Only some tasks succeeded

The first failed task’s Logs show what happened in that task. A preceding task may have succeeded, while later tasks are still blocked. After a correction has reached the instance, a new run records the result of the updated code. Earlier runs remain visible in the Grid view.

The example log shows [10, 20, 30], but no 60

That is the prepare_data log. In the same run, check whether calculate_total and report_total succeeded. Open report_total, then Logs, to find Total: 60. DAG runs and task logs on CloudFerro Cloud illustrates the result in the task log. A returned value of None after that printed message is normal for report_total.

Tasks stay queued

Tasks can wait while other DAGs occupy the available task slots. The instance status shows whether the service itself is healthy. The size and concurrency limits in Your first Workflows instance on CloudFerro Cloud determine how much work it can run together. More nodes do not raise those instance limits.

No separate result file appears in S3

The example prints to task logs; it does not create a separate output file. No empty file or folder needs to be created in advance. To save a dataset or report, the DAG needs a task that writes that output explicitly.

Missing task logs

Every run, task, and attempt has its own log object. Log message source details gives the S3 location for the selected log; the layout is explained in Object storage for Workflows on CloudFerro Cloud.

Missing logs can also result from an unavailable bucket or a key pair that no longer permits reading and writing. If credentials have changed, the instance needs the updated values. Support can help when storage access has been restored but logging has not recovered.

Support for instance errors

The statuses Error, Creating error, Deleting error, and Partially operational provide more details when hovered over. Contact Support opens the support route. An instance name, time, action, and displayed error help identify the event. For task failures, the DAG name, run identifier, task name, and relevant log message provide the context. Access tokens and secret keys are not needed in the report.

Still need help?
If this article doesn’t answer all of your questions, our support team will do their best to help you.
Contact support