sharky555
/
SM-AU
1
0
Fork 0
Code Issues 0 Pull Requests 0 Releases 0 Wiki Activity
Browse Source

detect 24bit and choose media type

master
sharky555 3 years ago
parent
353e78beae
commit
164f8ea0a2
7 changed files with 1302 additions and 1220 deletions
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +3
    -3
      .gitignore
  2. +76
    -72
      README.md
  3. +815
    -751
      autoupload.py
  4. +27
    -27
      json_data/config.json.example
  5. +257
    -243
      json_data/dictionary.json
  6. +7
    -7
      requirements.txt
  7. +117
    -117
      smpy.py

+ 3
- 3
.gitignore View File

@@ -1,4 +1,4 @@
__pycache__
config.json
*session.dat
__pycache__
config.json
*session.dat
*.torrent

+ 76
- 72
README.md View File

@@ -1,76 +1,76 @@
This project is **NOT** being maintained as i'd rather rewrite it from scratch.

## Overview
**JPS-AU** is a tool for automating the uploading process on jpopsuki.eu.
This is intended to be used alongside BugsPy.

**Features:**
- SM Client.
- FTP Support
- FLAC/MP3 Support.
- BugsPy .log Support.

**Installation:**
- Install requirements
```
pip install -r requirements.txt
```

## Command Usage
```
python autoupload.py {command_name} {ID/URL}
```
Command | Description | Example
-d, --debug | Provides additional information on upload for debugging purposes | `python autoupload.py -d`
-f, --freeleech | Enables freeleech (VIP+ Userclass Requirement) | `python autoupload.py -f -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]"`
-t, --tags | Add additional tags to upload, separated with comma | `python autoupload.py -t "korean, female.vocalist" -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]"`
-dir, --directory | Appoint directory used for torrent creation | `python autoupload.py -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]"`
-dry, --dryrun | Carries out all actions other than the upload itself.| `python autoupload.py -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]" -dry`
-i, --imageURL | Set the torrent cover URL | `python autoupload.py -f -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3] -i "https://someimageurl.com/someimage.jpg""`

## Config.json

- It's not recommended to use both local watch/download folders and ftp watch/download folders at the same time as it will result in seeding from 2 locations.
- If generate_tracklist is set to false the script will try and find BugsPy logs within the selected log folder, this will use the comments tag to find the log. For example if 204815 is within your comments tag, it will search your log folder for 204815.log and use the contents of this log for the album description.
- Similarly to generate_tracklist, if enable_release_description is 'true' the script will create a url to the Bugs.co.kr album webpage to display the source, this can easily be edited to suit your needs within the gatherdata() function

**credentials:**

Config | Description | Example
Username | SugoiMusic Username | Slyy
Password | SugoiMusic Password | Password
Passkey | SugoiMusic Passkey | Passkey

**local_prefs**

Config | Description | Example
log_directory | directory containing BugsPy log files | `Z:/Bugs/Logs`
generate_tracklist | enable tracklist generation | `true/false`
save_tracklist | write tracklist to .log and save in log folder | `true/false`
enable_release_description | post comments tag to release description | `true/false`
cover_name | name of cover with extension | `cover.jpg`
add_to_watch_folder | moves .torrent file to local watch folder | `true/false`
add_to_downloads_folder | moves torrent data to local downloads folder | `true/false`
local_watch_folder | directory of local watch folder | `Z:/watch/Transmission`
local_downloads_folder | directory of local downloads folder | `Z:/downloads`


**ftp_prefs:**

Config | Description | Example
enable_ftp | enable ftp mode, if enabled suggested to disable local watch and downloads folders | `true/false`
add_to_watch_folder | transfer .torrent file to watch folder on FTP server | `true/false`
add_to_downloads_folder | transfer torrent data to downloads folder on FTP server | `true/false`
ftp_server | url of ftp server | haze.seedhost.eu
ftp_username | username of ftp account | slyy
ftp_password | password of ftp account | password
ftp_watch_folder | directory of ftp watch folder | `/downloads/watch/transmission`
ftp_downloads_folder | directory of ftp downloads folder | `/downloads`


## Disclaimer
- The usage of this script **may be** illegal in your country. It's your own responsibility to inform yourself of Copyright Law.
This project is **NOT** being maintained as i'd rather rewrite it from scratch.
## Overview
**JPS-AU** is a tool for automating the uploading process on jpopsuki.eu.
This is intended to be used alongside BugsPy.
**Features:**
- SM Client.
- FTP Support
- FLAC/MP3 Support.
- BugsPy .log Support.
**Installation:**
- Install requirements
```
pip install -r requirements.txt
```
## Command Usage
```
python autoupload.py {command_name} {ID/URL}
```
Command | Description | Example
------------- | ------------- | -------------
-d, --debug | Provides additional information on upload for debugging purposes | `python autoupload.py -d`
-f, --freeleech | Enables freeleech (VIP+ Userclass Requirement) | `python autoupload.py -f -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]"`
-t, --tags | Add additional tags to upload, separated with comma | `python autoupload.py -t "korean, female.vocalist" -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]"`
-dir, --directory | Appoint directory used for torrent creation | `python autoupload.py -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]"`
-dry, --dryrun | Carries out all actions other than the upload itself.| `python autoupload.py -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3]" -dry`
-i, --imageURL | Set the torrent cover URL | `python autoupload.py -f -dir "Z:\Music\Korean\Ego\Ego - E [2020.01.02] [EP] [WEB-MP3] -i "https://someimageurl.com/someimage.jpg""`
## Config.json
- It's not recommended to use both local watch/download folders and ftp watch/download folders at the same time as it will result in seeding from 2 locations.
- If generate_tracklist is set to false the script will try and find BugsPy logs within the selected log folder, this will use the comments tag to find the log. For example if 204815 is within your comments tag, it will search your log folder for 204815.log and use the contents of this log for the album description.
- Similarly to generate_tracklist, if enable_release_description is 'true' the script will create a url to the Bugs.co.kr album webpage to display the source, this can easily be edited to suit your needs within the gatherdata() function
**credentials:**
Config | Description | Example
------------- | ------------- | -------------
Username | SugoiMusic Username | Slyy
Password | SugoiMusic Password | Password
Passkey | SugoiMusic Passkey | Passkey
**local_prefs**
Config | Description | Example
------------- | ------------- | -------------
log_directory | directory containing BugsPy log files | `Z:/Bugs/Logs`
generate_tracklist | enable tracklist generation | `true/false`
save_tracklist | write tracklist to .log and save in log folder | `true/false`
enable_release_description | post comments tag to release description | `true/false`
cover_name | name of cover with extension | `cover.jpg`
add_to_watch_folder | moves .torrent file to local watch folder | `true/false`
add_to_downloads_folder | moves torrent data to local downloads folder | `true/false`
local_watch_folder | directory of local watch folder | `Z:/watch/Transmission`
local_downloads_folder | directory of local downloads folder | `Z:/downloads`
**ftp_prefs:**
Config | Description | Example
------------- | ------------- | -------------
enable_ftp | enable ftp mode, if enabled suggested to disable local watch and downloads folders | `true/false`
add_to_watch_folder | transfer .torrent file to watch folder on FTP server | `true/false`
add_to_downloads_folder | transfer torrent data to downloads folder on FTP server | `true/false`
ftp_server | url of ftp server | haze.seedhost.eu
ftp_username | username of ftp account | slyy
ftp_password | password of ftp account | password
ftp_watch_folder | directory of ftp watch folder | `/downloads/watch/transmission`
ftp_downloads_folder | directory of ftp downloads folder | `/downloads`
## Disclaimer
- The usage of this script **may be** illegal in your country. It's your own responsibility to inform yourself of Copyright Law.

+ 815
- 751
autoupload.py
File diff suppressed because it is too large
View File


+ 27
- 27
json_data/config.json.example View File

@@ -1,27 +1,27 @@
{
"credentials": {
"username": "username",
"password": "password",
"passkey": "passkey"
},
"local_prefs": {
"log_directory": "D:/Bugs/BugsPy-master/logs",
"generate_tracklist": true,
"save_tracklist": false,
"enable_release_description": false,
"cover_name": "cover.jpg",
"add_to_watch_folder": false,
"add_to_downloads_folder": false,
"local_watch_folder": "C:/watch",
"local_downloads_folder": "C:/downloads"
},
"ftp_prefs": {
"enable_ftp": false,
"add_to_watch_folder": true,
"add_to_downloads_folder": true,
"ftp_server": "server url",
"ftp_username": "username",
"ftp_password": "password",
"ftp_watch_folder": "/downloads/watch/transmission",
"ftp_downloads_folder": "/downloads"
}
{
"credentials": {
"username": "username",
"password": "password",
"passkey": "passkey"
},
"local_prefs": {
"log_directory": "D:/Bugs/BugsPy-master/logs",
"generate_tracklist": true,
"save_tracklist": false,
"enable_release_description": false,
"cover_name": "cover.jpg",
"add_to_watch_folder": false,
"add_to_downloads_folder": false,
"local_watch_folder": "C:/watch",
"local_downloads_folder": "C:/downloads"
},
"ftp_prefs": {
"enable_ftp": false,
"add_to_watch_folder": true,
"add_to_downloads_folder": true,
"ftp_server": "server url",
"ftp_username": "username",
"ftp_password": "password",
"ftp_watch_folder": "/downloads/watch/transmission",
"ftp_downloads_folder": "/downloads"
}

+ 257
- 243
json_data/dictionary.json View File

@@ -1,244 +1,258 @@
{
"release_types": {
"Album": "Album",
"Single": "Single",
"EP": "Album",
"OST": "Album",
"싱글": "Single",
"EP(미니)": "Album",
"정규": "Album",
"컴필레이션": "Album",
"베스트": "Album",
"미니": "Album"
},
"genres": {
"R&B": "rnb",
"소울": "Soul",
"힙합": "hip.hop",
"랩": "Rap",
"영화": "Movie",
"로맨스": "Romance",
"드라마": "OST",
"TV 드라마": "OST",
"애니메이션": "anime",
"인디": "Indie",
"인디힙합": "Indie Hip-Hop",
"재즈 힙합": "Jazz-Hop",
"댄스 팝": "Dance",
"발라드": "Ballad",
"댄스": "Dance",
"포크 팝": "Folk",
"팝": "Pop",
"팝 락": "Pop.Rock",
"인디 락": "Indie.Rock",
"락": "Rock",
"메탈": "Metal",
"인디 팝": "Indie.Pop",
"일렉트로닉": "Electronic",
"일렉트로닉 팝": "Electro",
"인디일렉트로닉": "Indie.Electronic",
"신스 팝": "Synth-Pop",
"J-POP": "J-Pop",
"재즈": "Jazz",
"성인가요": "Trot",
"월드뮤직": "World Music",
"국악": "Traditional",
"종교": "Religious",
"CCM": "CCM",
"어린이": "Child",
"태교": "Taegyo",
"캐롤": "Christmas",
"트랩": "Trap",
"얼터너티브 팝": "Alternative.Pop",
"얼터너티브": "Alternative",
"뉴에이지": "New Age",
"켈틱": "Celtic",
"켈틱 퓨전": "Celtic.Fusion",
"퓨전": "Fusion",
"에스닉 퓨전": "Ethnic.Fusion",
"레게": "Reggae",
"댄스홀": "Dancehall",
"하우스": "House",
"트로트": "Trot",
"얼터너티브 락": "Alternative.Rock",
"덥": "Dub",
"싸이키델릭": "Psychedelic",
"인스트루멘탈 힙합": "Instrumental.Hip-Hop",
"인스트루멘탈": "Instrumental",
"클래식": "Classic",
"컨트리": "Country",
"종교음악": "Religious",
"전통음악": "Traditional",
"블루스": "Blues",
"라틴": "Latin",
"기타": "Other",
"기능성음악": "Functional",
"인디포크": "indie.folk",
"포크": "Folk",
"어쿠스틱": "Acoustic",
"Hip-Hop": "hip.hop"
},
"artist": {
"오아": "OA",
"이고": "Ego",
"ハルカトミユキ": null,
"琴音": null,
"下村陽子 × suis from ヨルシカ": null,
"川島ケイジ": null,
"裸体": null,
"空音": null,
"さかいゆう": null,
"美波": null,
"アルカラ": null,
"윤상": null,
"ブレッド & バター": null,
"Official髭男dism": null,
"優里": null,
"サニーデイ・サービス": null,
"ずっと真夜中でいいのに。": null,
"やなぎなぎ": null,
"米津玄師": null,
"梶浦由記": null,
"澁谷逆太郎": null,
"ポルカドットスティングレイ": null,
"김트와친구들": null,
"安斉かれん": null,
"坂口有望": null,
"空想委員会": null,
"ヨルシカ": null,
"向井太一": null,
"ペンギンラッシュ": null,
"黒子首": null,
"中島みゆき": null,
"ハリィさんとスイカくらぶ": null,
"堀込高樹": null,
"堀込泰行": null,
"スピラ・スピカ": null,
"17歳とベルリンの壁": null,
"天野月": null,
"ソールドシュガー": null,
"ナンカノユメ": null,
"ルルルルズ": null,
"東京事変": null,
"藍井エイル": null,
"阿部真央": null,
"赤いくらげ": null,
"週末CITY PLAY BOYZ": null,
"林 浩司": null,
"蒼山幸子": null,
"フラスコテーション": null,
"ゑんら": null,
"ハンブレッダーズ": null,
"鈴木このみ": null,
"みゆな": null,
"ビッケブランカ": null,
"めありー": null,
"キタニタツヤ": null,
"イロメガネ": null,
"ヤユヨ": null,
"ピロカルピン": null,
"ツユ": null,
"リリー楽綺団": null,
"山崎ハコ": null,
"いきものがかり": null,
"はるまきごはん": null,
"おくみずき": null,
"渣泥": null,
"竹渕慶": null,
"早見沙織": null,
"倖田來未": null,
"世武裕子": null,
"ラブリーサマーちゃん": null,
"SUPER☆GiRLS": null,
"österreich": null,
"フレデリック": null,
"ズーカラデル": null,
"神山羊": null,
"太田ひな": null,
"ヤバイTシャツ屋さん": null,
"當山みれい": null,
"大森靖子": null,
"大原櫻子": null,
"東京スカパラダイスオーケストラ": null,
"三月のパンタシア": null,
"雨のパレード": null,
"川崎鷹也": null,
"中島 美嘉": null,
"加藤ミリヤ": null,
"りぶ": null,
"雨ニマケテモ": null,
"三浦大知": null,
"コブクロ": null,
"ももいろクローバーZ": null,
"手嶌葵": null,
"Nao☆": null,
"尾崎裕哉": null,
"マーティ・フリードマン": null,
"幾田りら": null,
"山本彩": null,
"ビッケブランカ VS 岡崎体育": null,
"まるりとりゅうが": null,
"藤原さくら": null,
"藤井風": null,
"sicboy": "",
"LUCA & haruka nakamura": "arca",
"伊沢麻未": null,
"マカロニえんぴつ": null,
"チャラン・ポ・ランタン": null,
"鈴木瑛美子": null,
"神はサイコロを振らない": null,
"宇野実彩子": "AAA",
"ウルトラタワー": null,
"空白ごっこ": null,
"Cö shu Nie": null,
"くるり": null,
"流線形 & 一十三十一": null,
"清水翔太": null,
"あれくん": null,
"秋山黄色": null,
"웬디": "WENDY",
"瀧川ありさ": null,
"キリンジ": null,
"ユアネス": null,
"クレナズム": null,
"H△G": null,
"電音部": null,
"武藤彩未": null,
"中島美嘉": null,
"雫": null,
"坂本真綾": null,
"たかやん": null,
"布袋寅泰": null,
"アイラヴミー": null,
"ナナヲアカリ": null,
"福山雅治": null,
"Jacob&よみぃ": null,
"クミコ": null,
"リュックと添い寝ごはん": null,
"眉村ちあき": null,
"ちゃんみな & SKY-HI": null,
"関口シンゴ": null,
"角巻わため": null,
"Snail’s House": null,
"ロザリーナ": null,
"ニノミヤユイ": null,
"シド": null,
"森内寛樹": null,
"TK from 凛として時雨": null,
"スダンナユズユリー": null,
"ヤなことそっとミュート": null,
"宇徳敬子": null,
"Heavenstamp & ハハノシキュウ": null,
"비": null,
"ぼっちぼろまる": null,
"さなり": null,
"家入レオ": null,
"凛として時雨": null,
"緑黄色社会": null,
"あるゆえ": null,
"宮下 遊": null,
"夢乃ゆき": null,
"女王蜂": null,
"夜の本気ダンス": null
}
{
"release_types": {
"Album": "Album",
"Single": "Single",
"EP": "Album",
"OST": "Album",
"싱글": "Single",
"EP(미니)": "Album",
"정규": "Album",
"컴필레이션": "Album",
"베스트": "Album",
"미니": "Album"
},
"genres": {
"R&B": "rnb",
"소울": "Soul",
"힙합": "hip.hop",
"랩": "Rap",
"영화": "Movie",
"로맨스": "Romance",
"드라마": "OST",
"TV 드라마": "OST",
"애니메이션": "anime",
"인디": "Indie",
"인디힙합": "Indie Hip-Hop",
"재즈 힙합": "Jazz-Hop",
"댄스 팝": "Dance",
"발라드": "Ballad",
"댄스": "Dance",
"포크 팝": "Folk",
"팝": "Pop",
"팝 락": "Pop.Rock",
"인디 락": "Indie.Rock",
"락": "Rock",
"메탈": "Metal",
"인디 팝": "Indie.Pop",
"일렉트로닉": "Electronic",
"일렉트로닉 팝": "Electro",
"인디일렉트로닉": "Indie.Electronic",
"신스 팝": "Synth-Pop",
"J-POP": "J-Pop",
"재즈": "Jazz",
"성인가요": "Trot",
"월드뮤직": "World Music",
"국악": "Traditional",
"종교": "Religious",
"CCM": "CCM",
"어린이": "Child",
"태교": "Taegyo",
"캐롤": "Christmas",
"트랩": "Trap",
"얼터너티브 팝": "Alternative.Pop",
"얼터너티브": "Alternative",
"뉴에이지": "New Age",
"켈틱": "Celtic",
"켈틱 퓨전": "Celtic.Fusion",
"퓨전": "Fusion",
"에스닉 퓨전": "Ethnic.Fusion",
"레게": "Reggae",
"댄스홀": "Dancehall",
"하우스": "House",
"트로트": "Trot",
"얼터너티브 락": "Alternative.Rock",
"덥": "Dub",
"싸이키델릭": "Psychedelic",
"인스트루멘탈 힙합": "Instrumental.Hip-Hop",
"인스트루멘탈": "Instrumental",
"클래식": "Classic",
"컨트리": "Country",
"종교음악": "Religious",
"전통음악": "Traditional",
"블루스": "Blues",
"라틴": "Latin",
"기타": "Other",
"기능성음악": "Functional",
"인디포크": "indie.folk",
"포크": "Folk",
"어쿠스틱": "Acoustic",
"Hip-Hop": "hip.hop"
},
"artist": {
"오아": "OA",
"이고": "Ego",
"ハルカトミユキ": null,
"琴音": null,
"下村陽子 × suis from ヨルシカ": null,
"川島ケイジ": null,
"裸体": null,
"空音": null,
"さかいゆう": null,
"美波": null,
"アルカラ": null,
"윤상": null,
"ブレッド & バター": null,
"Official髭男dism": null,
"優里": null,
"サニーデイ・サービス": null,
"ずっと真夜中でいいのに。": null,
"やなぎなぎ": null,
"米津玄師": null,
"梶浦由記": null,
"澁谷逆太郎": null,
"ポルカドットスティングレイ": null,
"김트와친구들": null,
"安斉かれん": null,
"坂口有望": null,
"空想委員会": null,
"ヨルシカ": null,
"向井太一": null,
"ペンギンラッシュ": null,
"黒子首": null,
"中島みゆき": null,
"ハリィさんとスイカくらぶ": null,
"堀込高樹": null,
"堀込泰行": null,
"スピラ・スピカ": null,
"17歳とベルリンの壁": null,
"天野月": null,
"ソールドシュガー": null,
"ナンカノユメ": null,
"ルルルルズ": null,
"東京事変": null,
"藍井エイル": null,
"阿部真央": null,
"赤いくらげ": null,
"週末CITY PLAY BOYZ": null,
"林 浩司": null,
"蒼山幸子": null,
"フラスコテーション": null,
"ゑんら": null,
"ハンブレッダーズ": null,
"鈴木このみ": null,
"みゆな": null,
"ビッケブランカ": null,
"めありー": null,
"キタニタツヤ": null,
"イロメガネ": null,
"ヤユヨ": null,
"ピロカルピン": null,
"ツユ": null,
"リリー楽綺団": null,
"山崎ハコ": null,
"いきものがかり": null,
"はるまきごはん": null,
"おくみずき": null,
"渣泥": null,
"竹渕慶": null,
"早見沙織": null,
"倖田來未": null,
"世武裕子": null,
"ラブリーサマーちゃん": null,
"SUPER☆GiRLS": null,
"österreich": null,
"フレデリック": null,
"ズーカラデル": null,
"神山羊": null,
"太田ひな": null,
"ヤバイTシャツ屋さん": null,
"當山みれい": null,
"大森靖子": null,
"大原櫻子": null,
"東京スカパラダイスオーケストラ": null,
"三月のパンタシア": null,
"雨のパレード": null,
"川崎鷹也": null,
"中島 美嘉": null,
"加藤ミリヤ": null,
"りぶ": null,
"雨ニマケテモ": null,
"三浦大知": null,
"コブクロ": null,
"ももいろクローバーZ": null,
"手嶌葵": null,
"Nao☆": null,
"尾崎裕哉": null,
"マーティ・フリードマン": null,
"幾田りら": null,
"山本彩": null,
"ビッケブランカ VS 岡崎体育": null,
"まるりとりゅうが": null,
"藤原さくら": null,
"藤井風": null,
"sicboy": "",
"LUCA & haruka nakamura": "arca",
"伊沢麻未": null,
"マカロニえんぴつ": null,
"チャラン・ポ・ランタン": null,
"鈴木瑛美子": null,
"神はサイコロを振らない": null,
"宇野実彩子": "AAA",
"ウルトラタワー": null,
"空白ごっこ": null,
"Cö shu Nie": null,
"くるり": null,
"流線形 & 一十三十一": null,
"清水翔太": null,
"あれくん": null,
"秋山黄色": null,
"웬디": "WENDY",
"瀧川ありさ": null,
"キリンジ": null,
"ユアネス": null,
"クレナズム": null,
"H△G": null,
"電音部": null,
"武藤彩未": null,
"中島美嘉": null,
"雫": null,
"坂本真綾": null,
"たかやん": null,
"布袋寅泰": null,
"アイラヴミー": null,
"ナナヲアカリ": null,
"福山雅治": null,
"Jacob&よみぃ": null,
"クミコ": null,
"リュックと添い寝ごはん": null,
"眉村ちあき": null,
"ちゃんみな & SKY-HI": null,
"関口シンゴ": null,
"角巻わため": null,
"Snail’s House": null,
"ロザリーナ": null,
"ニノミヤユイ": null,
"シド": null,
"森内寛樹": null,
"TK from 凛として時雨": null,
"スダンナユズユリー": null,
"ヤなことそっとミュート": null,
"宇徳敬子": null,
"Heavenstamp & ハハノシキュウ": null,
"비": null,
"ぼっちぼろまる": null,
"さなり": null,
"家入レオ": null,
"凛として時雨": null,
"緑黄色社会": null,
"あるゆえ": null,
"宮下 遊": null,
"夢乃ゆき": null,
"女王蜂": null,
"夜の本気ダンス": null,
"藤末樹": null,
"崎山蒼志": null,
"草野華余子": null,
"マット・キャブ & MIYACHI": null,
"Maika Loubté": null,
"青山みつ紀": null,
"アイナ・ジ・エンド": null,
"須田景凪": null,
"あっこゴリラ": null,
"小林香織": null,
"L'Arc～en～Ciel": null,
"星街すいせい": null,
"高槻かなこ": null,
"Leeu & こゆき": null
}
}

+ 7
- 7
requirements.txt View File

@@ -1,7 +1,7 @@
bs4 == 0.0.1
langdetect == 1.0.7
mutagen == 1.42.0
requests == 2.22.0
torf == 2.1.0
tqdm == 4.39.0
html5lib
bs4 == 0.0.1
langdetect == 1.0.7
mutagen == 1.42.0
requests == 2.22.0
torf == 2.1.0
tqdm == 4.39.0
html5lib

+ 117
- 117
smpy.py View File

@@ -1,117 +1,117 @@
import os
import pickle
import datetime
from urllib.parse import urlparse
import requests
class MyLoginSession:
def __init__(self,
loginUrl,
loginData,
loginTestUrl,
loginTestString,
sessionFileAppendix='_session.dat',
maxSessionTimeSeconds=30 * 60,
proxies=None,
userAgent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1',
debug=False,
forceLogin=False,
**kwargs):
"""
save some information needed to login the session
you'll have to provide 'loginTestString' which will be looked for in the
responses html to make sure, you've properly been logged in
'proxies' is of format { 'https' : 'https://user:pass@server:port', 'http' : ...
'loginData' will be sent as post data (dictionary of id : value).
'maxSessionTimeSeconds' will be used to determine when to re-login.
"""
urlData = urlparse(loginUrl)
self.proxies = proxies
self.loginData = loginData
self.loginUrl = loginUrl
self.loginTestUrl = loginTestUrl
self.maxSessionTime = maxSessionTimeSeconds
self.sessionFile = urlData.netloc + sessionFileAppendix
self.userAgent = userAgent
self.loginTestString = loginTestString
self.debug = debug
self.login(forceLogin, **kwargs)
def modification_date(self, filename):
"""
return last file modification date as datetime object
"""
t = os.path.getmtime(filename)
return datetime.datetime.fromtimestamp(t)
def login(self, forceLogin=False, **kwargs):
"""
login to a session. Try to read last saved session from cache file. If this fails
do proper login. If the last cache access was too old, also perform a proper login.
Always updates session cache file.
"""
wasReadFromCache = False
if self.debug:
print('loading or generating session...')
if os.path.exists(self.sessionFile) and not forceLogin:
time = self.modification_date(self.sessionFile)
# only load if file less than 30 minutes old
lastModification = (datetime.datetime.now() - time).seconds
if lastModification < self.maxSessionTime:
with open(self.sessionFile, "rb") as f:
self.session = pickle.load(f)
wasReadFromCache = True
if self.debug:
print("loaded session from cache (last access %ds ago) "
% lastModification)
if not wasReadFromCache:
self.session = requests.Session()
self.session.headers.update({'user-agent': self.userAgent})
res = self.session.post(self.loginUrl, data=self.loginData,
proxies=self.proxies, **kwargs)
if self.debug:
print('created new session with login')
self.saveSessionToCache()
# test login
res = self.session.get(self.loginTestUrl)
if res.text.lower().find(self.loginTestString.lower()) < 0:
if self.debug:
print(res.text)
raise Exception("could not log into provided site '%s'"
" (did not find successful login string)"
% self.loginUrl)
def saveSessionToCache(self):
"""
save session to a cache file
"""
# always save (to update timeout)
with open(self.sessionFile, "wb") as f:
pickle.dump(self.session, f)
if self.debug:
print('updated session cache-file %s' % self.sessionFile)
def retrieveContent(self, url, method="get", postData=None, postDataFiles=None, **kwargs):
"""
return the content of the url with respect to the session.
If 'method' is not 'get', the url will be called with 'postData'
as a post request.
"""
if method == 'get':
res = self.session.get(url, proxies=self.proxies, **kwargs)
else:
res = self.session.post(url, data=postData, proxies=self.proxies, files=postDataFiles, **kwargs)
# the session has been updated on the server, so also update in cache
self.saveSessionToCache()
return res
import os
import pickle
import datetime
from urllib.parse import urlparse
import requests
class MyLoginSession:
def __init__(self,
loginUrl,
loginData,
loginTestUrl,
loginTestString,
sessionFileAppendix='_session.dat',
maxSessionTimeSeconds=30 * 60,
proxies=None,
userAgent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1',
debug=False,
forceLogin=False,
**kwargs):
"""
save some information needed to login the session
you'll have to provide 'loginTestString' which will be looked for in the
responses html to make sure, you've properly been logged in
'proxies' is of format { 'https' : 'https://user:pass@server:port', 'http' : ...
'loginData' will be sent as post data (dictionary of id : value).
'maxSessionTimeSeconds' will be used to determine when to re-login.
"""
urlData = urlparse(loginUrl)
self.proxies = proxies
self.loginData = loginData
self.loginUrl = loginUrl
self.loginTestUrl = loginTestUrl
self.maxSessionTime = maxSessionTimeSeconds
self.sessionFile = urlData.netloc + sessionFileAppendix
self.userAgent = userAgent
self.loginTestString = loginTestString
self.debug = debug
self.login(forceLogin, **kwargs)
def modification_date(self, filename):
"""
return last file modification date as datetime object
"""
t = os.path.getmtime(filename)
return datetime.datetime.fromtimestamp(t)
def login(self, forceLogin=False, **kwargs):
"""
login to a session. Try to read last saved session from cache file. If this fails
do proper login. If the last cache access was too old, also perform a proper login.
Always updates session cache file.
"""
wasReadFromCache = False
if self.debug:
print('loading or generating session...')
if os.path.exists(self.sessionFile) and not forceLogin:
time = self.modification_date(self.sessionFile)
# only load if file less than 30 minutes old
lastModification = (datetime.datetime.now() - time).seconds
if lastModification < self.maxSessionTime:
with open(self.sessionFile, "rb") as f:
self.session = pickle.load(f)
wasReadFromCache = True
if self.debug:
print("loaded session from cache (last access %ds ago) "
% lastModification)
if not wasReadFromCache:
self.session = requests.Session()
self.session.headers.update({'user-agent': self.userAgent})
res = self.session.post(self.loginUrl, data=self.loginData,
proxies=self.proxies, **kwargs)
if self.debug:
print('created new session with login')
self.saveSessionToCache()
# test login
res = self.session.get(self.loginTestUrl)
if res.text.lower().find(self.loginTestString.lower()) < 0:
if self.debug:
print(res.text)
raise Exception("could not log into provided site '%s'"
" (did not find successful login string)"
% self.loginUrl)
def saveSessionToCache(self):
"""
save session to a cache file
"""
# always save (to update timeout)
with open(self.sessionFile, "wb") as f:
pickle.dump(self.session, f)
if self.debug:
print('updated session cache-file %s' % self.sessionFile)
def retrieveContent(self, url, method="get", postData=None, postDataFiles=None, **kwargs):
"""
return the content of the url with respect to the session.
If 'method' is not 'get', the url will be called with 'postData'
as a post request.
"""
if method == 'get':
res = self.session.get(url, proxies=self.proxies, **kwargs)
else:
res = self.session.post(url, data=postData, proxies=self.proxies, files=postDataFiles, **kwargs)
# the session has been updated on the server, so also update in cache
self.saveSessionToCache()
return res

Write Preview
Loading…
Cancel
Save