Some jobs can be configured to read from or write to Google Cloud Storage (GCS).
You can configure a combination of Solr and cloud-based input or output, that is, you can read from GCS and then write to Solr or vice versa. However, you cannot configure multiple storage sources for input or multiple storage targets for output; only one can be configured for each.
See also Configure An Argo-Based Job to Access S3.
This procedure applies to these Argo jobs:
Content based Recommender
Evaluate QnA Pipeline
QnA Coldstart Training
QnA Supervised Training
For Spark jobs, see Configure A Spark-Based Job to Access Cloud Storage.
How to configure a job to access GCS
Gather the access key for your GCS account.
See the GCS documentation.
Create a Kubernetes secret:
kubectl create secret generic my-gcs-serviceaccount-key --from-file=<my-gcs-serviceaccount-key>.json --namespace <fusion-namespace>
In the job’s Cloud storage secret name field, enter the name of the secret for the GCS target as mounted in the Kubernetes namespace.
This is the name you specified in the previous step. In the example above, the secret name is
You can also find this name using
kubectl get secret -n <fusion-namespace>.
In the job’s Additional Parameters, add this parameter:
<name of the keyfile that is available when the GCS secret is mounted to the pod>
The file name may be different than the secret name. You can check using
kubectl get secret -n <fusion-namespace> <secretname> -o yaml.