google-cloud-storage - PermissionDeniedError：来自 /job:worker/replica:0/task:0

Question

我正在使用 google colaboratory 对 bert 进行预训练。我的 train_data 在谷歌云存储上，我的代码在 colab 上

IS_COLAB_BACKEND = 'COLAB_GPU' in os.environ  
if IS_COLAB_BACKEND:
    from google.colab import auth
    auth.authenticate_user()
model_saved_path = 'gs://zhongxing_train/model/roberta_pretrain_model2.ckpt'
corpus_paths = ['gs://zhongxing_train/corpus.0.tfrecord' ]
tpu_address = 'grpc://' + os.environ['COLAB_TPU_ADDR']

流动的是错误消息：

PermissionDeniedError                     Traceback (most recent call last)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)



1355     try:
-> 1356       return fn(*args)
   1357     except errors.OpError as e:

10 frames
PermissionDeniedError: From /job:worker/replica:0/task:0:
Error executing an HTTP request: HTTP response code 403 with body '{
  "error": {
    "code": 403,
    "message": "[e-mail] does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
    "errors": [
      {
        "message": "[e-mail] does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
        "domain": "global",
        "reason": "forbidden"
      }
    ]
  }
}
'
     when reading metadata of gs://zhongxing_train/corpus.0.tfrecord
     [[{{node MultiDeviceIteratorGetNextFromShard}}]]
     [[RemoteCall]]
     [[IteratorGetNextAsOptional]]

During handling of the above exception, another exception occurred:

PermissionDeniedError                     Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1368           pass
   1369       message = error_interpolation.interpolate(message, self._graph)
-> 1370       raise type(e)(node_def, op, message)
   1371 
   1372   def _extend_graph(self):

PermissionDeniedError: From /job:worker/replica:0/task:0:
Error executing an HTTP request: HTTP response code 403 with body '{
  "error": {
    "code": 403,
    "message": "[e-mail] does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
    "errors": [
      {
        "message": "[e-mail] does not have storage.objects.get access to zhongxing_train/corpus.0.tfrecord.",
        "domain": "global",
        "reason": "forbidden"
      }
    ]
  }
}
'
     when reading metadata of gs://zhongxing_train/corpus.0.tfrecord
     [[{{node MultiDeviceIteratorGetNextFromShard}}]]
     [[RemoteCall]]
     [[IteratorGetNextAsOptional]]

这是我的 colab 线

我想也许我需要在云存储上为我的 train_data 设置权限如何从谷歌云存储获得许可

标签： google-cloud-storagegoogle-colaboratorybert-language-model

Answer 1

您需要授予您的服务帐户storage.objects.get权限，您可以通过授予服务帐户至少Storage Object Viewer角色来做到这一点。

您可以使用 Cloud Console 或 Cloud SDK 来完成。