首页
学习
活动
专区
圈层
工具
发布
首页
学习
活动
专区
圈层
工具
MCP广场
社区首页 >问答首页 >将多个DBs转换为CSV

将多个DBs转换为CSV
EN

Stack Overflow用户
提问于 2016-05-22 14:03:37
回答 3查看 953关注 0票数 0

我有数千个dB文件需要转换为CSV文件。这可以通过一个简单的脚本/批处理文件来实现,即

代码语言:javascript
运行
复制
.open "Test.db"
.mode csv
.headers on.

我需要脚本来打开所有名称都不同的其他db文件,因为我不想为每个db文件编写上面的脚本,是否可以这样做?

EN

回答 3

Stack Overflow用户

回答已采纳

发布于 2016-05-24 13:30:56

sqlite3命令行shell允许使用命令行参数执行一些设置,因此您可以简单地为每个DB文件中的表执行一个简单的SELECT *

代码语言:javascript
运行
复制
for %%a in (*.db) do sqlite3 -csv -header "%%a" "select * from TableName" > %%~na.csv

(如果这不是批处理文件的一部分,而是直接从命令行运行,则必须将%%替换为%。)

票数 1
EN

Stack Overflow用户

发布于 2017-03-15 08:20:03

我创建了一个脚本,批处理将当前目录中的所有db-sqlite文件转换为CSV,称为“sqlite2csv”。它输出每个db-sqlite的每个表作为CSV文件,所以如果您有10个文件和3个表,您将得到30个CSV文件。希望它至少能作为你自己的脚本的起点。

代码语言:javascript
运行
复制
#!/bin/bash

# USAGE EXAMPLES :
# sqlite2csv
# - Will loop all sqlite files in the current directory, take the tables of
#   each of these sqlite files, and generate a CSV file per table.
#   E.g. If there are 10 sqlite files with 3 tables each, it will generate
#        30 CSV output files, each containing the data of one table.
#   The naming of the generated CSV files take from the original sqlite
#   file name, prepended with the name of the table.

# check for dependencies
if ! type "sqlite3" > /dev/null; then
    echo "[ERROR] SQLite binary not found."
    exit 1
fi

# define list of string tokens that an SQLite file type should contain
# the footprint for SQLite 3 is "SQLite 3.x database"
declare -a list_sqlite_tok
list_sqlite_tok+=( "SQLite" )
#list_sqlite_tok+=( "3.x" )
list_sqlite_tok+=( "database" )

# get a lis tof only files in current path
list_files=( $(find . -maxdepth 1 -type f) )

# loop the list of files
for f in ${!list_files[@]}; do
    # get current file
    curr_fname=${list_files[$f]}
    # get file type result
    curr_ftype=$(file -e apptype -e ascii -e encoding -e tokens -e cdf -e compress -e elf -e tar $curr_fname)
    # loop through necessary token and if one is not found then skip this file
    curr_isqlite=0
    for t in ${!list_sqlite_tok[@]}; do
        curr_tok=${list_sqlite_tok[$t]}
        # check if 'curr_ftype' contains 'curr_tok'
        if [[ $curr_ftype =~ $curr_tok ]]; then
            curr_isqlite=1
        else
            curr_isqlite=0
            break
        fi
    done
    # test if curr file was sqlite
    if (( ! $curr_isqlite )); then
        # if not, do not continue executung rest of script
        continue
    fi
    # print sqlite filename
    echo "[INFO] Found SQLite file $curr_fname, exporting tables..."
    # get tables of sqlite file in one line
    curr_tables=$(sqlite3 $curr_fname ".tables")
    # split tables line into an array
    IFS=$' ' list_tables=($curr_tables)
    # loop array to export each table
    for t in ${!list_tables[@]}; do
        curr_table=${list_tables[$t]}
        # strip unsafe characters as well as newline
        curr_table=$(tr '\n' ' ' <<< $curr_table)
        curr_table=$(sed -e 's/[^A-Za-z0-9._-]//g' <<< $curr_table) 
        # temporarily strip './' from filename
        curr_fname=${curr_fname//.\//}
        # build target CSV filename
        printf -v curr_csvfname "%s_%s.csv" $curr_table "$curr_fname"
        # put back './' to filenames
        curr_fname="./"$curr_fname
        curr_csvfname="./"$curr_csvfname
        # export current table to target CSV file
        sqlite3 -header -csv $curr_fname "select * from $curr_table;" > $curr_csvfname
        # log
        echo "[INFO] Exported table $curr_table in file $curr_csvfname"
    done
done
票数 2
EN

Stack Overflow用户

发布于 2019-01-17 12:04:57

我准备了一个简短的python脚本,它将从多个sqlite数据库中编写一个csv文件。

代码语言:javascript
运行
复制
#function for merging sqlite files to csv
def convert_sqlite_to_csv(inputFolder, ext, tableName):
    """ inputFolder - Folder where sqlite files are located. 
        ext - Extension of your sqlite file (eg. db, sqlite, sqlite3 etc.)
        tableName - table name from which you want to select the data.
    """
    csvWriter = csv.writer(open(inputFolder+'/output.csv', 'w', newline=''))
    for file1 in os.listdir(inputFolder):
        if file1.endswith('.'+ext):
            conn = sqlite3.connect(inputFolder+'/'+file1)
            cursor = conn.cursor()
            cursor.execute("SELECT * FROM "+tableName)
            rows = cursor.fetchall()
            for row in rows:
                csvWriter.writerow(row)
            continue
        else:
            continue

或者在下面的github链接上找到脚本,用于转换一个文件夹中的多个文件。

代码语言:javascript
运行
复制
python multiple_sqlite_files_tocsv.py -d <inputFolder> -e <extension> -t <tableName>

将数据输出到output.csv文件。

木星笔记本和巨蟒脚本都在github上。

https://github.com/darshanz/CombineMultipleSqliteToCsv

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/37375344

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档