S.O.N.G/touhouwiki-mirror
S.O.N.G/touhouwiki-mirror
3
0
Fork 0
Code Issues Pull requests Releases Activity

Update README, fix listing/go.sum

Browse source
This commit is contained in:
DataHoarder 2022-02-19 03:17:42 +01:00
parent af393f33cc
commit c438ab1f63
3 changed files with 17 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
README.md
View file

@ -10,9 +10,15 @@ Also includes data fetched under [data/](data/).
* `$ go run . -path data/` for running a server. Opens on port 8777.
# Content server features
* Access the entire content tree by path (ex. `/pages/54742.wiki` or `/pages_by_name/RETRO_FUTURE_GIRLS.wiki`)
Small Golang server that loads all albums in an index at startup and is used to serve files/results to queries.
* Access the entire content tree by path
* Access page by id ex. `/pages/54742.wiki`
* Access page by name ex. `/pages_by_name/RETRO_FUTURE_GIRLS.wiki`
* List files in a directory (ex. `/pages/` or `/pageindex/Arrangement_CDs/`)
* Query database by either any album title or album catalog number (ex. `/search?type=title&query=マジコカタストロフィ` or `/search?type=title&query=STAL-1302`)
* Query database by either any album title or album catalog number
* Title ex. `/search?type=title&query=マジコカタストロフィ`
* Catalog ex. `/search?type=title&query=STAL-1302`
* Values here are normalized so special characters or uppercase are changed. It deals with Unicode too.
* Bare CDDB emulator for `query` and `read` commands. Used to query by track count + length of album.
* query cmd ex. `/cddb?cmd=cddb+query+730dec08` (no need to provide TOC, but can be provided)

2
go.sum Normal file
View file

@ -0,0 +1,2 @@
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=

13
listing.sh
View file

@ -35,7 +35,7 @@ getQueryResultCmContinue () {
}
fetchCategoryRequestPage () {
RESULT_PATH="${PAGE_PATH}${2}/${1}.json"
RESULT_PATH="${PAGE_PATH}${2//[\/]/_}/${1}.json"
doCategoryRequest "${2}" "${3}" > .listing.tmp.json
if [[ $(isValidQueryResult .listing.tmp.json) == "yes" ]]; then
jq . .listing.tmp.json > "${RESULT_PATH}"
@ -47,11 +47,12 @@ fetchCategoryRequestPage () {
return 1
}
CATEGORY="${1//[ \/]/_}"
CATEGORY="${1//[ ]/_}"
PATHCATEGORY="${CATEGORY//[\/]/_}"
PAGE=1
CMCONTINUE=""
mkdir -p "${PAGE_PATH}${CATEGORY}"
mkdir -p "${PAGE_PATH}${PATHCATEGORY}"
while [[ 1 ]]; do
@ -60,21 +61,21 @@ while [[ 1 ]]; do
echo "Fetch page $PAGE"
# Page does not exist, try fetch
if [[ ! -f "${PAGE_PATH}${CATEGORY}/${PAGE}.json" ]]; then
if [[ ! -f "${PAGE_PATH}${PATHCATEGORY}/${PAGE}.json" ]]; then
RESULT_PATH=$(fetchCategoryRequestPage "${PAGE}" "${CATEGORY}" "${CMCONTINUE}")
if [[ "$RESULT_PATH" == "" ]]; then
break
fi
CMCONTINUE=$(getQueryResultCmContinue "${RESULT_PATH}")
# Is it last page? If so fetch
elif [[ ! -f "${PAGE_PATH}${CATEGORY}/${NEXT_PAGE}.json" ]]; then
elif [[ ! -f "${PAGE_PATH}${PATHCATEGORY}/${NEXT_PAGE}.json" ]]; then
RESULT_PATH=$(fetchCategoryRequestPage "${PAGE}" "${CATEGORY}" "${CMCONTINUE}")
if [[ "$RESULT_PATH" == "" ]]; then
break
fi
CMCONTINUE=$(getQueryResultCmContinue "${RESULT_PATH}")
else
CMCONTINUE=$(getQueryResultCmContinue "${PAGE_PATH}${CATEGORY}/${PAGE}.json")
CMCONTINUE=$(getQueryResultCmContinue "${PAGE_PATH}${PATHCATEGORY}/${PAGE}.json")
fi
if [[ "$CMCONTINUE" == "" || "$CMCONTINUE" == "null" ]]; then