延續上一篇 Image Host Transfer Program with ChatGPT V1 (failed)，歷經失敗後我終於成功了，對，不是我們。
credit: https://makeameme.org/meme/me-and-chatgpt
Couclusion First
因為對話廢話太多，就先說結論了。每次來都改一堆的情況真的讓我頗痛苦，所以後來受不了就自己去找 google API 來看了，突然覺得舒服多了。所以這邊看到我只有問了幾次。而且因為大部分程式碼 (例如找連結、下載之類的) 其實在上一篇已經寫好，所以我幾乎從原本的版本複製過來，然後把 API 換成 google drive。啪就可以跑了，看到本篇文的同時目前圖床也轉換完畢。完整程式碼可以 右轉 GitHub 查看。
經過此次教訓，寫程式上我可能就沒什麼動力去問 ChatGPT 怎麼寫了。小程式 (例如已經有一堆答案的 leetcode，或是排序演算法) 可能還行，稍微大一點就不行了 (此圖床轉換程式，V1: 246 行，V2: 313 行) 更何況是破千破萬行的大型程式 (雖然讀個研究所是可能幾百行就 OK)。它可能還是較適合其他用途，例如產生幹話和產生幹話。
Communicate with ChatGPT: Python program for downloading images and uploading to Google Drive.
Prompt attempt
👨💻 Human:
2 / 2
Implement a python program to download all image from a given markdown file locally and then upload them to specific folder of google drive
please split the program into functions, a function should as small as possible
please always use f-string
please throw error when handling exception
please add a main funtion to it, the main do: 1. find all image link and caption in a markdown file.
2. download all image based on the url.
3. upload downloaded images named by their caption to a folder of google drive named by the markdown filename.
4. replace old url to new url in the markdown file.
5. delete all temporary files
🤖 Assistant:
Sure, here is an implementation of the program with the specified requirements:
import os
import re
import shutil
import requests
from google.oauth2.credentials import Credentials
from google.auth.transport.requests import Request
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# Define the regular expression pattern to match the image URLs and captions
image_regex = r"!\[([\w\s]+)\]\((http[s]?:\/\/[\w\/\-\.]+)\)"
def find_image_links(markdown_file):
"""
Given a markdown file, finds all image links and captions in the file.
Args:
markdown_file (str): Path to the markdown file.
Returns:
A list of tuples, each tuple containing the image caption and URL.
"""
with open(markdown_file, "r") as f:
content = f.read()
return re.findall(image_regex, content)
def download_image(image_url, caption):
"""
Given an image URL, downloads the image locally.
Args:
image_url (str): URL of the image to download.
caption (str): Caption of the image, used as the filename for the downloaded image.
Returns:
The path to the downloaded image file.
"""
response = requests.get(image_url, stream=True)
if response.status_code == 200:
filename = f"{caption}.{image_url.split('.')[-1]}"
with open(filename, "wb") as f:
response.raw.decode_content = True
shutil.copyfileobj(response.raw, f)
return os.path.abspath(filename)
else:
raise ValueError(f"Failed to download image from {image_url}")
def upload_to_drive(filename, folder_name):
"""
Given a filename and a folder name, uploads the file to the specified Google Drive folder.
Args:
filename (str): The path to the file to upload.
folder_name (str): The name of the Google Drive folder to upload the file to.
Returns:
The URL of the uploaded file on Google Drive.
"""
creds = None
if os.path.exists("token.json"):
creds = Credentials.from_authorized_user_file("token.json")
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
raise ValueError("Failed to authenticate with Google Drive API.")
drive_service = build("drive", "v3", credentials=creds)
try:
# Find the specified folder by name
query = f"name = '{folder_name}' and mimeType = 'application/vnd.google-apps.folder'"
results = drive_service.files().list(q=query, spaces="drive", fields="nextPageToken, files(id, name)").execute()
items = results.get("files", [])
if not items:
raise ValueError(f"No folder found with the name {folder_name}")
folder_id = items[0]["id"]
# Upload the file to the specified folder
file_metadata = {"name": os.path.basename(filename), "parents": [folder_id]}
media = {"media_body": filename}
file = drive_service.files().create(body=file_metadata, media_body=media, fields="id, webContentLink").execute()
return file.get("webContentLink")
except HttpError as error:
raise ValueError(f"Failed to upload file to Google Drive: {error}")
def replace_image_links(markdown_file, old_links, new_links):
"""
Given a markdown file, replaces old image links with new image links.
Args:
markdown_file (str): Path to the markdown file.
old_links (list): List of
Prompt attempt
👨💻 Human:
continue, and start from def replace_image_links
🤖 Assistant:
Sure, here is the continued implementation of the program:
def replace_image_links(markdown_file, old_links, new_links):
"""
Given a markdown file, replaces old image links with new image links.
Args:
markdown_file (str): Path to the markdown file.
old_links (list): List of tuples containing the old image captions and URLs.
new_links (list): List of tuples containing the new image captions and URLs.
Returns:
None.
"""
with open(markdown_file, "r") as f:
content = f.read()
for old_link, new_link in zip(old_links, new_links):
content = content.replace(f"![{old_link[0]}]({old_link[1]})", f"![{new_link[0]}]({new_link[1]})")
with open(markdown_file, "w") as f:
f.write(content)
def main(markdown_file, folder_name):
"""
The main function of the program, which downloads all images in the specified markdown file,
uploads them to a specified Google Drive folder, replaces old image links with new image links,
and deletes temporary files.
Args:
markdown_file (str): Path to the markdown file.
folder_name (str): Name of the Google Drive folder to upload the images to.
Returns:
None.
"""
# Find all image links and captions in the markdown file
image_links = find_image_links(markdown_file)
# Download all images and keep track of their local paths and captions
image_paths = []
image_captions = []
for caption, url in image_links:
path = download_image(url, caption)
image_paths.append(path)
image_captions.append(caption)
# Upload all images to the specified Google Drive folder and keep track of their URLs and captions
image_urls = []
for path, caption in zip(image_paths, image_captions):
url = upload_to_drive(path, folder_name)
image_urls.append((caption, url))
# Replace all old image links with new image links in the markdown file
replace_image_links(markdown_file, image_links, image_urls)
# Delete all temporary files
for path in image_paths:
os.remove(path)
This program defines four functions:
find_image_links: Given a markdown file, finds all image links and captions in the file.download_image: Given an image URL, downloads the image locally.upload_to_drive: Given a filename and a folder name, uploads the file to the specified Google Drive folder.replace_image_links: Given a markdown file, replaces old image links with new image links.
The main function uses these functions to implement the program as specified in the prompt. It first finds all image links and captions in the markdown file, then downloads all images and uploads them to the specified Google Drive folder, replaces the old image links with the new image links in the markdown file, and finally deletes all temporary files.
Prompt attempt
👨💻 Human:
3 / 3
In your upload_to_drive function, please use service_account to build the google drive service
🤖 Assistant:
Sure, here is the updated upload_to_drive function that uses a service account to build the Google Drive service:
from google.oauth2 import service_account
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
def upload_to_drive(filename, folder_name):
"""
Given a filename and a folder name, uploads the file to the specified Google Drive folder.
Args:
filename (str): Name of the file to upload.
folder_name (str): Name of the Google Drive folder to upload the file to.
Returns:
str: The URL of the uploaded file.
"""
# Authenticate and build the Google Drive service
credentials = service_account.Credentials.from_service_account_file('path/to/service_account.json')
drive_service = build('drive', 'v3', credentials=credentials)
# Get the ID of the folder to upload the file to
folder_id = None
query = f"name='{folder_name}' and mimeType='application/vnd.google-apps.folder' and trashed = false"
response = drive_service.files().list(q=query, fields='nextPageToken, files(id, name)').execute()
files = response.get('files', [])
if not files:
raise ValueError(f'Folder "{folder_name}" not found.')
else:
folder_id = files[0]['id']
# Upload the file to the folder
file_metadata = {'name': filename, 'parents': [folder_id]}
media = MediaFileUpload(filename, resumable=True)
file = None
try:
file = drive_service.files().create(
body=file_metadata,
media_body=media,
fields='id, webContentLink').execute()
except HttpError as error:
print(f'An error occurred: {error}')
raise
return file.get('webContentLink')
This updated implementation uses the service_account.Credentials class to authenticate with Google Drive using a service account. It then builds the Google Drive service using the build function from the googleapiclient.discovery module, passing in the credentials object.
The rest of the function is the same as before - it gets the ID of the specified Google Drive folder, then uploads the specified file to that folder using the drive_service.files().create() method, and returns the web content link of the uploaded file.
Note that you will need to replace 'path/to/service_account.json' in the code with the actual path to your service account JSON file.
Prompt attempt
👨💻 Human:
2 / 2
Use ArgumentParser to input markdown file path and credentials path in your main function
🤖 Assistant:
Sure, here’s an updated version of the main function that uses ArgumentParser to take input for the markdown file path and the credentials path:
import os
import re
import argparse
import urllib.request
import shutil
from google.oauth2 import service_account
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
from googleapiclient.http import MediaFileUpload
def download_image(url):
"""
Given a URL, downloads the image from the URL and returns the filename.
Args:
url (str): The URL of the image to download.
Returns:
str: The filename of the downloaded image.
"""
# Download the image
with urllib.request.urlopen(url) as response:
data = response.read()
filename = url.split('/')[-1]
# Save the image to a file
with open(filename, 'wb') as f:
f.write(data)
return filename
def upload_to_drive(filename, folder_name, credentials_path):
"""
Given a filename, a folder name, and a credentials path, uploads the file to the specified Google Drive folder.
Args:
filename (str): Name of the file to upload.
folder_name (str): Name of the Google Drive folder to upload the file to.
credentials_path (str): Path to the service account credentials file.
Returns:
str: The URL of the uploaded file.
"""
# Authenticate and build the Google Drive service
credentials = service_account.Credentials.from_service_account_file(credentials_path)
drive_service = build('drive', 'v3', credentials=credentials)
# Get the ID of the folder to upload the file to
folder_id = None
query = f"name='{folder_name}' and mimeType='application/vnd.google-apps.folder' and trashed = false"
response = drive_service.files().list(q=query, fields='nextPageToken, files(id, name)').execute()
files = response.get('files', [])
if not files:
raise ValueError(f'Folder "{folder_name}" not found.')
else:
folder_id = files[0]['id']
# Upload the file to the folder
file_metadata = {'name': filename, 'parents': [folder_id]}
media = MediaFileUpload(filename, resumable=True)
file = None
try:
file = drive_service.files().create(
body=file_metadata,
media_body=media,
fields='id, webContentLink').execute()
except HttpError as error:
print(f'An error occurred: {error}')
raise
return file.get('webContentLink')
def replace_image_links(markdown_file):
"""
Given a markdown file, finds all image links and captions, downloads the images, and replaces the links with the uploaded image URLs.
Args:
markdown_file (str): The path to the markdown file.
Returns:
str: The updated markdown text.
"""
with open(markdown_file, 'r') as f:
text = f.read()
# Find all image links and captions in the markdown text
pattern = r"!\[(.*?)\]\((.*?)\)"
matches = re.findall(pattern, text)
# Download the images and upload them to Google Drive
for caption, url in matches:
filename = download_image(url)
uploaded_url = upload_to_drive(filename, os.path.splitext(markdown_file)[0], args.credentials)
os.remove(filename)
# Replace the old image link with the new uploaded image link
text = text.replace(f'', f'![{caption
Prompt attempt
👨💻 Human:
continue
🤖 Assistant:
Sure, here’s the updated main function that uses ArgumentParser:
def main():
# Parse command line arguments
parser = argparse.ArgumentParser(description='Download images from a markdown file and upload them to Google Drive.')
parser.add_argument('markdown_file', type=str, help='Path to the markdown file.')
parser.add_argument('credentials', type=str, help='Path to the service account credentials file.')
args = parser.parse_args()
# Replace image links in the markdown file and upload the images to Google Drive
updated_text = replace_image_links(args.markdown_file)
# Save the updated markdown file
with open(args.markdown_file, 'w') as f:
f.write(updated_text)
if __name__ == '__main__':
main()
Now you can call this program from the command line, passing in the markdown file path and the credentials path like this:
python program.py my_markdown.md /path/to/credentials.json
This will replace the image links in my_markdown.md with the uploaded image URLs and save the updated file. It will also upload the downloaded images to Google Drive, naming the folder after the markdown filename.
Prompt attempt
👨💻 Human:
your main function did not follow the requirement: the main do: 1. find all image link and caption in a markdown file.
2. download all image based on the url.
3. upload downloaded images named by their caption to a folder of google drive named by the markdown filename.
4. replace old url to new url in the markdown file.
5. delete all temporary files
🤖 Assistant:
My apologies, you’re right. Here’s the updated main function that follows the requirements:
def main():
# Parse command line arguments
parser = argparse.ArgumentParser(description='Download images from a markdown file and upload them to Google Drive.')
parser.add_argument('markdown_file', type=str, help='Path to the markdown file.')
parser.add_argument('credentials', type=str, help='Path to the service account credentials file.')
args = parser.parse_args()
# Find all image links and captions in the markdown file
image_links = find_image_links(args.markdown_file)
# Download all images based on the URLs
downloaded_images = download_images(image_links)
# Upload the downloaded images to Google Drive
folder_name = os.path.splitext(os.path.basename(args.markdown_file))[0]
drive_service = build_drive_service(args.credentials)
folder_id = create_folder(drive_service, folder_name)
uploaded_images = []
for image in downloaded_images:
uploaded_image = upload_to_drive(drive_service, folder_id, image['path'], image['caption'])
uploaded_images.append(uploaded_image)
# Replace old URLs with new URLs in the markdown file
updated_text = replace_image_links(args.markdown_file, uploaded_images)
# Save the updated markdown file
with open(args.markdown_file, 'w') as f:
f.write(updated_text)
# Delete all temporary files
for image in downloaded_images:
os.remove(image['path'])
This should perform all the necessary steps outlined in the requirements:
- Find all image links and captions in the markdown file using the
find_image_linksfunction. - Download all images based on the URLs using the
download_imagesfunction. - Upload the downloaded images to Google Drive using the
upload_to_drivefunction, naming them according to their captions and storing them in a folder named after the markdown filename using thecreate_folderfunction. - Replace old URLs with new URLs in the markdown file using the
replace_image_linksfunction. - Save the updated markdown file and delete all temporary files.
Prompt attempt
👨💻 Human:
there is no build_drive_service and create_folder functions in your main function
🤖 Assistant:
You’re right, apologies for the confusion. Here’s the updated main function that includes the build_drive_service and create_folder functions:
def main():
# Parse command line arguments
parser = argparse.ArgumentParser(description='Download images from a markdown file and upload them to Google Drive.')
parser.add_argument('markdown_file', type=str, help='Path to the markdown file.')
parser.add_argument('credentials', type=str, help='Path to the service account credentials file.')
args = parser.parse_args()
# Find all image links and captions in the markdown file
image_links = find_image_links(args.markdown_file)
# Download all images based on the URLs
downloaded_images = download_images(image_links)
# Upload the downloaded images to Google Drive
folder_name = os.path.splitext(os.path.basename(args.markdown_file))[0]
drive_service = build_drive_service(args.credentials)
folder_id = create_folder(drive_service, folder_name)
uploaded_images = []
for image in downloaded_images:
uploaded_image = upload_to_drive(drive_service, folder_id, image['path'], image['caption'])
uploaded_images.append(uploaded_image)
# Replace old URLs with new URLs in the markdown file
updated_text = replace_image_links(args.markdown_file, uploaded_images)
# Save the updated markdown file
with open(args.markdown_file, 'w') as f:
f.write(updated_text)
# Delete all temporary files
for image in downloaded_images:
os.remove(image['path'])
def build_drive_service(credentials_file):
with open(credentials_file, 'r') as f:
credentials = json.load(f)
credentials = service_account.Credentials.from_service_account_info(credentials)
service = build('drive', 'v3', credentials=credentials)
return service
def create_folder(service, folder_name):
folder_metadata = {
'name': folder_name,
'mimeType': 'application/vnd.google-apps.folder'
}
folder = service.files().create(body=folder_metadata, fields='id').execute()
return folder.get('id')
The build_drive_service function reads the service account credentials from a JSON file, builds a Credentials object, and returns a Google Drive API service object authenticated with the credentials.
The create_folder function creates a folder with a given name in the user’s Google Drive and returns the ID of the created folder.
Exported on 2023年4月15日 下午2:49.