在Google App Engine Flex上运行时，使用Java将文件从Google Storage Bucket移动到Google Drive
在Google App Engine Flex上运行时，使用Java将文件从Google Storage Bucket移动到Google Drive
提问于 2018-03-27 02:00:48
我正在Google App Engine Flex环境中运行一个应用程序。我的目标是允许从应用程序向Google Drive上传文件。由于App Engine没有文件系统(或者是我真正理解的文件系统)，Google的文档中提到要上传到Storage Bucket：
https://cloud.google.com/appengine/docs/flexible/java/using-cloud-storage
我有很好的代码工作-上传到存储桶。Google Drive的代码也运行得很好。
我的问题是-如何将存储桶文件上传到Google Drive？
我发现了一些帖子，它们建议我必须从Storage Bucket下载文件，据我所知，这些文件必须保存到文件系统中，然后由Google Drive代码拾取，然后上传到Google Drive。
copy file from Google Drive to Google Cloud Storage within Google
How to download a file from Google Cloud Storage with Java?
我需要做的就是从Storage Bucket下载中设置一个java.io.File对象。我有从Google Storage代码返回的blob.getMediaLink() -可以使用它而不是下载吗？
Google存储代码：
public static String sendFileToBucket(InputStream fileStream, String fileName) throws IOException {
Logger.info("GoogleStorage: sendFileToBucket: Starting...");
GoogleCredential credential = null;
String credentialsFileName = "";
Storage storage = null;
Blob blob = null;
List<Acl> acls = null;
// credential = authorize();
// storage = StorageOptions.getDefaultInstance().getService();
try {
Logger.info("GoogleStorage: sendFileToBucket: Getting credentialsFileName path...");
credentialsFileName = Configuration.root().getString("google.storage.credentials.file");
Logger.info("GoogleStorage: sendFileToBucket: credentialsFileName = " + credentialsFileName);
Logger.info("GoogleStorage: sendFileToBucket: Setting InputStream...");
InputStream in = GoogleStorage.class.getClassLoader().getResourceAsStream(credentialsFileName);
if (in == null) {
Logger.info("GoogleStorage: sendFileToBucket: InputStream is null");
}
Logger.info("GoogleStorage: sendFileToBucket: InputStream set...");
try {
storage = StorageOptions.newBuilder().setCredentials(ServiceAccountCredentials.fromStream(in)).build()
.getService();
} catch (StorageException se) {
System.out.println("--- START ERROR WITH SETTING STORAGE OBJECT ---");
se.printStackTrace();
System.out.println("--- END ERROR WITH SETTING STORAGE OBJECT ---");
}
// Modify access list to allow all users with link to read file
acls = new ArrayList<>();
acls.add(Acl.of(Acl.User.ofAllUsers(), Acl.Role.READER));
try {
Logger.info("GoogleStorage: sendFileToBucket: Setting Blob object...");
blob = storage.create(BlobInfo.newBuilder(BUCKET_NAME, fileName).setAcl(acls).build(), fileStream);
Logger.info("GoogleStorage: sendFileToBucket: Blob Object set...");
} catch (StorageException se) {
System.out.println("--- START ERROR WITH SETTING BLOB OBJECT ---");
se.printStackTrace();
System.out.println("--- END ERROR WITH SETTING BLOB OBJECT ---");
}
} catch (IOException ex) {
System.out.println("--- START ERROR SENDFILETOBUCKET ---");
ex.printStackTrace();
System.out.println("--- END ERROR SENDFILETOBUCKET ---");
}
Logger.info("GoogleStorage: sendFileToBucket: blob.getMediaLink() = " + blob.getMediaLink());
return blob.getMediaLink();
}Google Drive代码：
public static String uploadFile(java.io.File file, String folderIDToFind) throws IOException {
String fileID = "";
String fileName = "";
try {
Logger.info("GoogleDrive: uploadFile: Starting File Upload...");
// Build a new authorized API client service.
Drive service = getDriveService();
Logger.info("GoogleDrive: uploadFile: Completed Drive Service...");
// Set the folder...
String folderID = Configuration.root().getString("google.drive.folderid");
Logger.info("GoogleDrive: uploadFile: Folder ID = " + folderID);
String folderIDToUse = getSubfolderID(service, folderID, folderIDToFind);
String fullFilePath = file.getAbsolutePath();
Logger.info("GoogleDrive: uploadFile: Full File Path: " + fullFilePath);
File fileMetadata = new File();
// Let's see what slashes exist to get the correct file name...
if (fullFilePath.contains("/")) {
fileName = StringControl.rightBack(fullFilePath, "/");
} else {
fileName = StringControl.rightBack(fullFilePath, "\\");
}
String fileContentType = getContentType(fileName);
Logger.info("GoogleDrive: uploadFile: File Content Type: " + fileContentType);
fileMetadata.setName(fileName);
Logger.info("GoogleDrive: uploadFile: File Name = " + fileName);
Logger.info("GoogleDrive: uploadFile: Setting the folder...");
fileMetadata.setParents(Collections.singletonList(folderIDToUse));
Logger.info("GoogleDrive: uploadFile: Folder set...");
// Team Drive settings...
fileMetadata.set("supportsTeamDrives", true);
FileContent mediaContent = new FileContent(fileContentType, file);
File fileToUpload = service.files().create(fileMetadata, mediaContent).setSupportsTeamDrives(true)
.setFields("id, parents").execute();
fileID = fileToUpload.getId();
Logger.info("GoogleDrive: uploadFile: File ID: " + fileID);
} catch (Exception ex) {
System.out.println(ex.toString());
ex.printStackTrace();
}
Logger.info("GoogleDrive: uploadFile: Ending File Upload...");
return fileID;
}在上面的帖子中，我看到了如何获取Storage Bucket文件--但我将其转到了java.io.file对象：
How to download a file from Google Cloud Storage with Java?
感谢你的帮助。
回答 2
Stack Overflow用户
回答已采纳
发布于 2018-03-30 03:57:49
非常感谢@Ying Li，我终于有了一个解决方案。我最终使用上传到存储存储桶后返回的blob.getMediaLink() java.io.File创建了一个临时URL。下面是我创建的方法：
public static java.io.File createFileFromURL(String fileURL, String fileName) throws IOException {
java.io.File tempFile = null;
try {
URL url = new URL(fileURL);
Logger.info("GoogleControl: createFileFromURL: url = " + url);
Logger.info("GoogleControl: createFileFromURL: fileName = " + fileName);
String filePrefix = StringControl.left(fileName, ".") + "_";
String fileExt = "." + StringControl.rightBack(fileName, ".");
Logger.info("GoogleControl: createFileFromURL: filePrefix = " + filePrefix);
Logger.info("GoogleControl: createFileFromURL: fileExt = " + fileExt);
tempFile = java.io.File.createTempFile(filePrefix, fileExt);
tempFile.deleteOnExit();
FileUtils.copyURLToFile(url, tempFile);
Logger.info("GoogleControl: createFileFromURL: tempFile name = " + tempFile.getName());
} catch (Exception ex) {
System.out.println(ex.toString());
ex.printStackTrace();
}
return tempFile;
}因此，完整的类如下所示：
package google;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import org.apache.commons.io.FileUtils;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.FileContent;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.util.store.FileDataStoreFactory;
import com.google.api.services.drive.Drive;
import com.google.api.services.drive.DriveScopes;
import com.google.api.services.drive.model.File;
import com.google.api.services.drive.model.FileList;
import com.google.auth.oauth2.*;
import com.google.cloud.storage.*;
import controllers.GlobalUtilities.StringControl;
import play.Configuration;
import play.Logger;
/**
* @author Dan Zeller
*
*/
public class GoogleControl {
private static final String APPLICATION_NAME = "PTP";
private static final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
private static HttpTransport HTTP_TRANSPORT;
private static final String BUCKET_NAME = Configuration.root().getString("google.storage.bucket.name");
public static final String FILE_PREFIX = "stream2file";
public static final String FILE_SUFFIX = ".tmp";
static {
try {
HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
} catch (Throwable t) {
t.printStackTrace();
System.exit(1);
}
}
@SuppressWarnings("deprecation")
public static GoogleCredential authorize() throws IOException {
GoogleCredential credential = null;
String credentialsFileName = "";
try {
Logger.info("GoogleControl: authorize: Starting...");
Logger.info("GoogleControl: authorize: Getting credentialsFileName path...");
credentialsFileName = Configuration.root().getString("google.drive.credentials.file");
Logger.info("GoogleControl: authorize: credentialsFileName = " + credentialsFileName);
Logger.info("GoogleControl: authorize: Setting InputStream...");
InputStream in = GoogleControl.class.getClassLoader().getResourceAsStream(credentialsFileName);
if (in == null) {
Logger.info("GoogleControl: authorize: InputStream is null");
}
Logger.info("GoogleControl: authorize: InputStream set...");
Logger.info("GoogleControl: authorize: Setting credential...");
credential = GoogleCredential.fromStream(in, HTTP_TRANSPORT, JSON_FACTORY)
.createScoped(Collections.singleton(DriveScopes.DRIVE));
} catch (IOException ex) {
System.out.println(ex.toString());
System.out.println("Could not find file " + credentialsFileName);
ex.printStackTrace();
}
Logger.info("GoogleControl: authorize: Ending...");
return credential;
}
public static java.io.File createFileFromURL(String fileURL, String fileName) throws IOException {
java.io.File tempFile = null;
try {
URL url = new URL(fileURL);
Logger.info("GoogleControl: createFileFromURL: url = " + url);
Logger.info("GoogleControl: createFileFromURL: fileName = " + fileName);
String filePrefix = StringControl.left(fileName, ".") + "_";
String fileExt = "." + StringControl.rightBack(fileName, ".");
Logger.info("GoogleControl: createFileFromURL: filePrefix = " + filePrefix);
Logger.info("GoogleControl: createFileFromURL: fileExt = " + fileExt);
tempFile = java.io.File.createTempFile(filePrefix, fileExt);
tempFile.deleteOnExit();
FileUtils.copyURLToFile(url, tempFile);
Logger.info("GoogleControl: createFileFromURL: tempFile name = " + tempFile.getName());
} catch (Exception ex) {
System.out.println(ex.toString());
ex.printStackTrace();
}
return tempFile;
}
public static void downloadFile(String fileID) {
// Set the drive service...
Drive service = null;
try {
service = getDriveService();
} catch (IOException e) {
e.printStackTrace();
}
OutputStream outputStream = new ByteArrayOutputStream();
try {
service.files().get(fileID).executeMediaAndDownloadTo(outputStream);
} catch (IOException e) {
e.printStackTrace();
}
}
public static String getContentType(String filePath) throws Exception {
String type = "";
try {
Path path = Paths.get(filePath);
type = Files.probeContentType(path);
System.out.println(type);
} catch (Exception ex) {
System.out.println(ex.toString());
ex.printStackTrace();
}
return type;
}
public static Drive getDriveService() throws IOException {
Logger.info("GoogleControl: getDriveService: Starting...");
GoogleCredential credential = null;
Drive GoogleControl = null;
try {
credential = authorize();
Logger.info("GoogleControl: getDriveService: Credentials set...");
try {
GoogleControl = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential)
.setApplicationName(APPLICATION_NAME).build();
} catch (Exception ex) {
System.out.println(ex.toString());
ex.printStackTrace();
}
} catch (IOException ex) {
System.out.println(ex.toString());
ex.printStackTrace();
}
return GoogleControl;
}
public static String getPath() {
String s = GoogleControl.class.getName();
int i = s.lastIndexOf(".");
if (i > -1)
s = s.substring(i + 1);
s = s + ".class";
System.out.println("Class Name: " + s);
Object testPath = GoogleControl.class.getResource(s);
System.out.println("Current Path: " + testPath);
return "";
}
public static String getSubfolderID(Drive service, String parentFolderID, String folderKeyToGet) {
// We need to see if the folder exists based on the ID...
String folderID = "";
Boolean foundFolder = false;
FileList result = null;
File newFolder = null;
// Set the drive query...
String driveQuery = "mimeType='application/vnd.google-apps.folder' and '" + parentFolderID
+ "' in parents and name='" + folderKeyToGet + "' and trashed=false";
try {
result = service.files().list().setQ(driveQuery).setIncludeTeamDriveItems(true).setSupportsTeamDrives(true)
.execute();
} catch (IOException e) {
e.printStackTrace();
}
for (File folder : result.getFiles()) {
System.out.printf("Found folder: %s (%s)\n", folder.getName(), folder.getId());
foundFolder = true;
folderID = folder.getId();
}
if (foundFolder != true) {
// Need to create the folder...
File fileMetadata = new File();
fileMetadata.setName(folderKeyToGet);
fileMetadata.setTeamDriveId(parentFolderID);
fileMetadata.set("supportsTeamDrives", true);
fileMetadata.setMimeType("application/vnd.google-apps.folder");
fileMetadata.setParents(Collections.singletonList(parentFolderID));
try {
newFolder = service.files().create(fileMetadata).setSupportsTeamDrives(true).setFields("id, parents")
.execute();
} catch (IOException e) {
e.printStackTrace();
}
// Send back the folder ID...
folderID = newFolder.getId();
System.out.println("Folder ID: " + newFolder.getId());
}
return folderID;
}
@SuppressWarnings("deprecation")
public static java.io.File sendFileToBucket(InputStream fileStream, String fileName) throws Exception {
Logger.info("GoogleControl: sendFileToBucket: Starting...");
GoogleCredential credential = null;
String credentialsFileName = "";
String outputFileName = "";
Storage storage = null;
Blob blob = null;
List<Acl> acls = null;
java.io.File returnFile = null;
try {
Logger.info("GoogleControl: sendFileToBucket: Getting credentialsFileName path...");
credentialsFileName = Configuration.root().getString("google.storage.credentials.file");
Logger.info("GoogleControl: sendFileToBucket: credentialsFileName = " + credentialsFileName);
Logger.info("GoogleControl: sendFileToBucket: Setting InputStream...");
InputStream in = GoogleControl.class.getClassLoader().getResourceAsStream(credentialsFileName);
if (in == null) {
Logger.info("GoogleControl: sendFileToBucket: InputStream is null");
}
Logger.info("GoogleControl: sendFileToBucket: InputStream set...");
try {
storage = StorageOptions.newBuilder().setCredentials(ServiceAccountCredentials.fromStream(in)).build()
.getService();
} catch (Exception se) {
System.out.println("--- START ERROR WITH SETTING STORAGE OBJECT ---");
se.printStackTrace();
System.out.println("--- END ERROR WITH SETTING STORAGE OBJECT ---");
}
// Modify access list to allow all users with link to read file
acls = new ArrayList<>();
acls.add(Acl.of(Acl.User.ofAllUsers(), Acl.Role.READER));
try {
Logger.info("GoogleControl: sendFileToBucket: Setting Blob object...");
blob = storage.create(BlobInfo.newBuilder(BUCKET_NAME, fileName).setAcl(acls).build(), fileStream);
Logger.info("GoogleControl: sendFileToBucket: Blob Object set...");
} catch (Exception se) {
System.out.println("--- START ERROR WITH SETTING BLOB OBJECT ---");
se.printStackTrace();
System.out.println("--- END ERROR WITH SETTING BLOB OBJECT ---");
}
Logger.info("GoogleControl: sendFileToBucket: blob.getMediaLink() = " + blob.getMediaLink());
// Let's build a java.io.file to send back...
returnFile = createFileFromURL(blob.getMediaLink(), fileName);
} catch (Exception ex) {
System.out.println("--- START ERROR SENDFILETOBUCKET ---");
ex.printStackTrace();
System.out.println("--- END ERROR SENDFILETOBUCKET ---");
}
return returnFile;
// return outputFileName;
}
public static String uploadFile(java.io.File file, String folderIDToFind) throws IOException {
String fileID = "";
String fileName = "";
try {
Logger.info("GoogleControl: uploadFile: Starting File Upload...");
// Build a new authorized API client service.
Drive service = getDriveService();
Logger.info("GoogleControl: uploadFile: Completed Drive Service...");
// Set the folder...
String folderID = Configuration.root().getString("google.drive.folderid");
Logger.info("GoogleControl: uploadFile: Folder ID = " + folderID);
String folderIDToUse = getSubfolderID(service, folderID, folderIDToFind);
String fullFilePath = file.getAbsolutePath();
Logger.info("GoogleControl: uploadFile: Full File Path: " + fullFilePath);
File fileMetadata = new File();
// Let's see what slashes exist to get the correct file name...
if (fullFilePath.contains("/")) {
fileName = StringControl.rightBack(fullFilePath, "/");
} else {
fileName = StringControl.rightBack(fullFilePath, "\\");
}
String fileContentType = getContentType(fileName);
Logger.info("GoogleControl: uploadFile: File Content Type: " + fileContentType);
fileMetadata.setName(fileName);
Logger.info("GoogleControl: uploadFile: File Name = " + fileName);
Logger.info("GoogleControl: uploadFile: Setting the folder...");
fileMetadata.setParents(Collections.singletonList(folderIDToUse));
Logger.info("GoogleControl: uploadFile: Folder set...");
// Team Drive settings...
fileMetadata.set("supportsTeamDrives", true);
FileContent mediaContent = new FileContent(fileContentType, file);
File fileToUpload = service.files().create(fileMetadata, mediaContent).setSupportsTeamDrives(true)
.setFields("id, parents").execute();
fileID = fileToUpload.getId();
Logger.info("GoogleControl: uploadFile: File ID: " + fileID);
} catch (Exception ex) {
System.out.println(ex.toString());
ex.printStackTrace();
}
Logger.info("GoogleControl: uploadFile: Ending File Upload...");
return fileID;
}
}下面是开始这一切的代码片段：
try {
Logger.info("AdultPTPController: Starting File Upload...");
File file = null;
File fileFinal = null;
String fileName = "";
String fileContentType = "";
String filePath = "";
String fileID = "";
String bucketFilePath = "";
String bucketFileName = "";
// String folderIDToFind =
// adultDemo.getNew_Provider_Id().toString();
String folderIDToFind = adultDemo.getLegacy_Provider_Id().toString();
Http.MultipartFormData<File> formData = request().body().asMultipartFormData();
if (formData != null) {
Http.MultipartFormData.FilePart<File> filePart = formData.getFile("fileAttach");
if (filePart != null) {
fileName = filePart.getFilename().trim();
// Is there a file?
if (!fileName.equals("") && fileName != null) {
Logger.info("AdultPTPController: File Name = " + fileName);
fileContentType = filePart.getContentType();
file = filePart.getFile();
long size = Files.size(file.toPath());
String fullFilePath = file.getPath();
InputStream fileStream = new FileInputStream(fullFilePath);
// Send the file/multipart content to the storage
// bucket...
Logger.info("AdultPTPController: Sending file to GoogleStorage - sendFileToBucket");
File bucketFile = GoogleControl.sendFileToBucket(fileStream, fileName);
//File bucketFile = null;
Logger.info("AdultPTPController: File Sent to GoogleStorage - sendFileToBucket");
// Send the file to Google Drive...
if (bucketFile != null) {
// Upload the file and return the file ID...
Logger.info("AdultPTPController: File is not null, sending to uploadFile...");
bucketFileName = bucketFile.getName();
Logger.info("AdultPTPController: bucketFileName = " + bucketFileName);
fileID = GoogleControl.uploadFile(bucketFile, folderIDToFind);
Logger.info("AdultPTPController: File ID = " + fileID);
if (!fileID.equals("")) {
// Success...
// Create a file record...
FileUpload fileUpload = new FileUpload();
// Create a unique ID...
fileUpload.setFileKey(fileUpload.createFileKey());
// Set the needed fields...
fileUpload.setRecordID(adultDemo.getLegacy_Provider_Id().toString());
fileUpload.setRecordKey(adultPTP.getPtpkey());
fileUpload.setCreatedBy(user.getFullname());
fileUpload.setCreatedByEmail(user.getEmail());
fileUpload.setCreatedByKey(user.getUserkey());
fileUpload.setCreatedDate(GlobalUtilities.getCurrentLocalDateTime());
fileUpload.setDateCreatedDisplay(
GlobalUtilities.getStringDate(fileUpload.getCreatedDate()));
// Set the file info...
fileUpload.setFileID(fileID);
fileUpload.setFileName(fileName);
fileUpload.setFilePath(filePath);
String folderID = Configuration.root().getString("google.drive.folderid");
fileUpload.setFolderID(folderID);
// Set the URL...
String urlString = Configuration.root().getString("server.hostname");
String fullURL = urlString + "/downloadfile?ptpKey=" + adultPTP.getPtpkey() + "&fileID="
+ fileID;
fileUpload.setFileURL(fullURL);
fileUpload.save();
}
}
}
}
}
} catch (Exception e) {
e.printStackTrace();
}我希望这对下一个人有帮助。
Stack Overflow用户
发布于 2018-03-27 07:52:26
您需要从GCS (Google Cloud Storage)将文件下载到本地计算机，然后让Java代码拾取该文件并将其发送到Google Drive。
可以使用代码的第二部分。但是您的第一部分实际上是将一个文件上传到GCS，然后获取一个服务URL。我不认为你想要这样。
相反，请查看downloading from GCS，然后运行代码的第二部分，并将该文件上传到Google Drive。
===
如果您使用的是App Engine，并且不想使用本地机器进行上传，您可以尝试以下方法：
Read from GCS and get the file to outputStream in the HTTP response。然后，您可以使用upload the file to Google Drive。有一些代码需要添加，您需要确保您获得的字节流被转换为java.io.File。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/49497778
复制相关文章
相似问题