Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix ard day page #1012

Closed
wants to merge 1,372 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1372 commits
Select commit Hold shift + click to select a range
b978d20
Fix link to architecture overview in readme
Nicklas2751 Oct 28, 2021
cff9a5b
Refactor the ARD crawler
Nicklas2751 Oct 29, 2021
aad098f
Add missing preview enable
Nicklas2751 Oct 29, 2021
3515c25
Fix maven tests
Nicklas2751 Oct 29, 2021
28047f9
Fix maven javadoc with enable-preview
Nicklas2751 Oct 29, 2021
c1cfcce
Fix maven javadoc with enable-preview
Nicklas2751 Oct 29, 2021
12f48a0
Merge pull request #773 from mediathekview/refactorings
TheSasch Oct 29, 2021
514a477
Merge pull request #772 from mediathekview/architecture-documentation
TheSasch Oct 29, 2021
d6c4b0f
Fix filmlist tests by using new new filmlists
Nicklas2751 Oct 31, 2021
6e3248b
#141 add testcase for nbsp encoding
pidoubleyou Oct 31, 2021
0e4f8a9
Update to JUnit5. First Step with Vintage Engine
TheSasch Nov 6, 2021
09c14b2
#775 phoenix: fix basename not available anymore
pidoubleyou Nov 18, 2021
0c822ac
kika: additional topic pages
pidoubleyou Dec 4, 2021
1eaa068
arte: refactor video deserialization
pidoubleyou Dec 12, 2021
b9e76a9
Updated log4j CVE-2021-44228
alex1702 Dec 13, 2021
635e8af
arte: refactor video deserialization
pidoubleyou Dec 16, 2021
4e996dd
Merge pull request #782 from mediathekview/feature/arte_languages
pidoubleyou Dec 21, 2021
877e7a8
update dependencies
pidoubleyou Jan 6, 2022
d339b27
Merge pull request #789 from mediathekview/dep_update_dev
pidoubleyou Jan 6, 2022
818186e
funk: decision to load subpages made in task not in deserializer
pidoubleyou Jan 6, 2022
cbc21ab
funk: support vertical resolutions
pidoubleyou Jan 15, 2022
a595d6f
Merge pull request #790 from mediathekview/feature/funk_fixes
pidoubleyou Jan 15, 2022
603d460
fix sonar error
pidoubleyou Jan 15, 2022
21d2bde
fix maximum subpages config
pidoubleyou Jan 23, 2022
5a10450
fix maximum subpages config
pidoubleyou Jan 23, 2022
76e4284
Merge pull request #793 from mediathekview/feature/funk_fixes2
pidoubleyou Jan 23, 2022
6ee0ef7
#794 VOF-STA is french film with german subtitle -> list in ARTE.DE
pidoubleyou Jan 27, 2022
48c7aaa
ARTE: fix days url
pidoubleyou Jan 30, 2022
d14cef1
ARTE: crawl most recent and last chance video lists
pidoubleyou Jan 30, 2022
94cbb23
ARTE: update subcategory url
pidoubleyou Jan 30, 2022
1c2499d
#796 fix duration of arte trailers
pidoubleyou Feb 5, 2022
9565a79
refactor to match master requirements
pidoubleyou Feb 6, 2022
e0a3578
add test for ArdTopicPageTask
pidoubleyou Feb 19, 2022
d010117
fix ard funk: m3u8 files containing relative urls
pidoubleyou Mar 6, 2022
2204632
Merge pull request #802 from mediathekview/feature/800
pidoubleyou Mar 6, 2022
e6c1eb8
ignore QXADRM and QXBDRM
pidoubleyou Mar 6, 2022
aa60ce7
#807 fix ARTE Geolocation EUR_DE_FR
pidoubleyou Mar 13, 2022
dff15fc
#810 add another teaser element
pidoubleyou Mar 13, 2022
35f8a4c
fix videoUrl contains only filename
pidoubleyou Mar 20, 2022
a5e2bc1
Merge pull request #813 from mediathekview/feature/812
pidoubleyou Mar 20, 2022
6be62de
add protocol to urls without "://"
pidoubleyou Mar 27, 2022
10ee3fc
#812 fix another srf bug with m3u8-files containing only filenames
pidoubleyou Mar 27, 2022
83e8c14
#812 fix another srf bug with m3u8-files containing only filenames
pidoubleyou Mar 27, 2022
ef1a689
add test for last segment
pidoubleyou Mar 27, 2022
d1e430e
Merge pull request #816 from mediathekview/feature/812
pidoubleyou Mar 27, 2022
15bd0f5
#812 remove drm videos
pidoubleyou Apr 3, 2022
ccc3996
#821 kika: fix geo
pidoubleyou Apr 18, 2022
958a029
#829 fix NumberFormatException:
pidoubleyou May 22, 2022
502ec4c
Merge pull request #830 from mediathekview/feature/829
pidoubleyou May 22, 2022
f102f7c
#834 handle uhd urls and use lowest hd url for normal if normal is mi…
pidoubleyou Aug 6, 2022
72bc46d
Merge pull request #837 from mediathekview/feature/834
pidoubleyou Aug 6, 2022
331e08c
#835 orf uses audio description urls of film class
pidoubleyou Aug 6, 2022
5ae8c62
#835 entry is valid if audio description or sign language url exists
pidoubleyou Aug 6, 2022
a07d185
#835 srf uses audio description urls of film class
pidoubleyou Aug 6, 2022
81a0fde
Merge pull request #838 from mediathekview/feature/835
pidoubleyou Aug 6, 2022
032007e
zdf add support of sign language videos
pidoubleyou Aug 7, 2022
8a66bdd
Merge pull request #842 from mediathekview/feature/827
pidoubleyou Aug 7, 2022
572c731
Fix wrong color for OkHttp
Nicklas2751 Oct 29, 2021
72dd0b8
Fix wrong color for OkHttp
Nicklas2751 Oct 29, 2021
67b8c81
Upgrade maven dependencies, remove useless Maven repository deploy
Nicklas2751 Oct 20, 2022
bb2e5bc
Merge pull request #848 from mediathekview/feature/upgradeDependencies
TheSasch Oct 20, 2022
8489402
Merge branch 'develop' into feature/junit5
TheSasch Oct 20, 2022
52b5331
Updated Dependency Version to best one without CVEs
TheSasch Oct 20, 2022
aa373f7
Merge pull request #774 from mediathekview/feature/junit5
TheSasch Oct 20, 2022
84539a7
Create a GitHub Action to create a pre-release for every develop commit
Nicklas2751 Oct 20, 2022
a4955be
Switch to fabric8 docker-maven-plugin
Nicklas2751 Oct 20, 2022
f25fea3
Fix GitHub artifacts for develop pre-release
Nicklas2751 Oct 20, 2022
1fbd286
Fix GitHub artifact path for THIRD-PARTY licenses
Nicklas2751 Oct 20, 2022
2ba2bdc
Remove test branch for develop pre-release deploy
Nicklas2751 Oct 20, 2022
788adc8
Updated BR Unit Tests to JUnit5 with AssertJ
TheSasch Oct 22, 2022
c99cfca
Updated to JUnit5 and used assertJ for assertions
ForBai Oct 23, 2022
1511ae3
Added second test, updated to JUnit5 and used assertJ for assertions
ForBai Oct 23, 2022
2cfee5c
update DW crawler to new API
codingPF Oct 23, 2022
24bdddf
Added sort pom plugin & sorted pom
Nicklas2751 Oct 22, 2022
cf15c3b
Fix logging error
Nicklas2751 Oct 22, 2022
1d3dd25
Fix sonar issues
Nicklas2751 Oct 23, 2022
f925fe5
call optional.isPresent before accessing the value
codingPF Oct 23, 2022
e8597fe
fix some code smell
codingPF Oct 23, 2022
218d46f
move branch
codingPF Oct 23, 2022
4705786
Merge pull request #856 from mediathekview/feature/cleanup
TheSasch Oct 23, 2022
1b1a99f
Updated Funk Unit Tests to JUnit5 with AssertJ
TheSasch Oct 23, 2022
2247d90
kika dev
codingPF Oct 30, 2022
770306a
Merge remote-tracking branch 'origin/develop' into dwfix
codingPF Oct 30, 2022
9070240
remove old source
codingPF Oct 30, 2022
2d0f169
split methods and add ratelimiter
codingPF Oct 31, 2022
32991c3
move urlbase to constants
codingPF Nov 1, 2022
5b6600b
undo not needed changes
codingPF Nov 1, 2022
01606df
fix sonar
codingPF Nov 1, 2022
c7a5f84
code smells
codingPF Nov 1, 2022
6e9bf83
typo
codingPF Nov 1, 2022
b7694fc
updates from review
codingPF Nov 6, 2022
f7eb515
fix some code smells
codingPF Nov 7, 2022
975973b
Merge pull request #857 from mediathekview/dwfix
codingPF Nov 10, 2022
6659f34
Sonarcloud bugs
codingPF Nov 10, 2022
da4e2fa
interrupt thread
codingPF Nov 10, 2022
770ad03
code smells
codingPF Nov 10, 2022
0d51906
code smells
codingPF Nov 10, 2022
43a32f8
remove old kika src, only use video page
codingPF Dec 14, 2022
27b78ac
Merge branch 'develop' into devkika
codingPF Dec 14, 2022
feda055
Merge pull request #859 from mediathekview/devkika
codingPF Dec 14, 2022
30dd887
fix scheduletask url
codingPF Dec 16, 2022
8b0ee9a
dw fix video resolutions
pidoubleyou Dec 18, 2022
1d78249
Merge pull request #863 from mediathekview/dwfix
pidoubleyou Dec 18, 2022
7b7646d
Merge pull request #862 from mediathekview/develop-sonarcloud-bugs
codingPF Dec 18, 2022
a8c1c9b
Merge pull request #864 from mediathekview/dev-fix-srf
codingPF Dec 18, 2022
414f480
reconstruct website for all entries
codingPF Dec 20, 2022
7f6b382
Merge pull request #866 from mediathekview/dev-fix-kika-website
codingPF Dec 20, 2022
0fc8c70
added funk core service translation
codingPF Dec 20, 2022
bcfbf92
Merge pull request #868 from mediathekview/dev-fix-funk
codingPF Dec 20, 2022
027de69
funk: fix double /api in url
pidoubleyou Dec 26, 2022
e6f5d9a
phoenix: fix missing title #869
pidoubleyou Dec 26, 2022
60c6998
fix missing hd url
pidoubleyou Jan 28, 2023
e3dbdd4
add another hd url
pidoubleyou Jan 29, 2023
d88ff8b
add method to get attribute value as String
pidoubleyou Feb 15, 2023
bea1faf
supports quality fhd and uhd
pidoubleyou Feb 15, 2023
bdd2b3e
use android-client instead of desktop browser client
pidoubleyou Feb 15, 2023
c54fdc5
optimize hd urls
pidoubleyou Feb 15, 2023
0966082
fix test
pidoubleyou Feb 15, 2023
8b107b8
Merge pull request #881 from mediathekview/feature/zdf_hd
pidoubleyou Feb 15, 2023
16d7d26
extend exception handling
pidoubleyou Feb 19, 2023
cc376eb
add language specific subtitle urls
pidoubleyou Feb 26, 2023
61fd25c
Merge pull request #885 from mediathekview/feature/884
pidoubleyou Feb 26, 2023
c1d84d3
fix missing hosts in some funk urls
pidoubleyou Mar 12, 2023
f8fec96
fix url parameter which do contain equals (e.g. SRF)
codingPF Jul 21, 2023
6227a65
remove constantly changing parameter from video url
codingPF Jul 21, 2023
970f8da
fix junit testcase
codingPF Jul 24, 2023
4dd0770
Merge pull request #907 from mediathekview/bugfix/develop904
pidoubleyou Aug 1, 2023
ddea42e
add ignoreFilmFilter
codingPF Aug 13, 2023
5805c65
Sonar cloud
codingPF Aug 13, 2023
c3e5a3a
junit coverrage and external files
codingPF Aug 13, 2023
e511370
code smell
codingPF Aug 19, 2023
a6c9f4a
Merge pull request #913 from mediathekview/dev/ignoreFilmFilter
codingPF Aug 19, 2023
9d7edce
add livestream filmlist import
codingPF Aug 20, 2023
3d595c5
add junit test for all formats
codingPF Aug 20, 2023
5aeab96
fix config init
codingPF Aug 20, 2023
d65c3c6
remove parameter from url extract from m3u8 playlist
codingPF Oct 2, 2023
2fc9a91
Sonarcloud fixes
codingPF Oct 2, 2023
372eadb
Merge pull request #926 from mediathekview/bugfix/923dev
codingPF Oct 2, 2023
0626c6c
Updated KikaApiTopicDeserializerTest to JUnit5
ForBai Oct 1, 2023
d598f9a
Updated KikaApiVideoInfoDeserializerTest to JUnit5
ForBai Oct 1, 2023
df6ad4d
Fix name in KikaApiTopicDeserializerTest.java
ForBai Oct 1, 2023
75d9d58
Fix name KikaApiVideoInfoDeserializerTest.java
ForBai Oct 1, 2023
d86a115
remove br crawler
pidoubleyou Oct 3, 2023
ff4de2b
remove br crawler
pidoubleyou Oct 3, 2023
6a05708
remove br crawler
pidoubleyou Oct 3, 2023
bb04fd1
remove BR crawler
pidoubleyou Oct 3, 2023
1989db1
update to Junit5
codingPF Oct 3, 2023
b805eb3
set default livestream file path to root
codingPF Oct 3, 2023
4b36765
typo
codingPF Oct 13, 2023
7fe4fa4
Merge pull request #927 from mediathekview/dev/livestreamImport
codingPF Oct 13, 2023
dc43ded
init importfilmlists
codingPF Oct 13, 2023
2c1af04
use crawler fileSize (like all the others do)
codingPF Nov 1, 2023
28de909
maxtime is in min and should be fine to check every min
codingPF Nov 1, 2023
db29465
add film available check
codingPF Nov 1, 2023
a8e197c
remove NexxCloudSessionId test until fixed
codingPF Nov 1, 2023
c8f11d9
move methods to mlib
codingPF Nov 1, 2023
14ab0a8
undo local change
codingPF Nov 1, 2023
fbf5c34
remove (old) importfilmlist configuration items
codingPF Nov 1, 2023
357c75e
add messages and fix java doc
codingPF Nov 1, 2023
813935b
do not remove livestreams
codingPF Nov 1, 2023
4f8060d
sonarcloud
codingPF Nov 1, 2023
cb44dfa
Merge pull request #939 from mediathekview/dev/importFilmlists
codingPF Nov 3, 2023
1460c7a
update configurationmanager to support configuration file name
codingPF Nov 5, 2023
5a5e59c
change to configurable pool size
codingPF Nov 8, 2023
402a0b8
write new and old format json
codingPF Nov 8, 2023
16e2afa
align classpath config and file config
codingPF Nov 8, 2023
642bfcb
Merge pull request #940 from mediathekview/feature/configfilename
codingPF Nov 8, 2023
769698e
add option to provide args to mserver docker
codingPF Nov 10, 2023
e91dacd
Move ratelimiter to crawler
codingPF Nov 12, 2023
b1af62d
#938 use url optimizers in kika crawler
pidoubleyou Nov 12, 2023
a9db335
ratelimiter for crawler calls, added exception for duplicates
codingPF Nov 15, 2023
5fb7547
threadsafe
codingPF Nov 15, 2023
1913d72
#938 use url optimizers in kika crawler
pidoubleyou Nov 12, 2023
74ae034
undo
codingPF Nov 16, 2023
5c46be8
typo
codingPF Nov 16, 2023
8a0a937
Merge branch 'develop' into feature/ratelimiter
codingPF Nov 16, 2023
f35a682
Merge pull request #948 from mediathekview/feature/ratelimiter
codingPF Nov 16, 2023
a3893ff
add resolution typo, log duplicates, add audiodescription to videos
codingPF Nov 17, 2023
7c7dc2f
no incremental mode - depht to be controlled by maximumSubpages
codingPF Nov 18, 2023
a301a50
add rejected log and remove logging (since already in subtask)
codingPF Nov 18, 2023
2d6eb39
fix DW to always have normal url, fix testcase
codingPF Nov 18, 2023
a4dd4d8
#938 optimize wdr urls
pidoubleyou Nov 21, 2023
9e44db2
#938 optimize wdr urls
pidoubleyou Nov 21, 2023
5a91a39
fix unknown duration "Sek."
pidoubleyou Dec 3, 2023
0c3cb99
search more episodes
pidoubleyou Jan 6, 2024
d2cf832
process more episodes only in topic search
pidoubleyou Jan 7, 2024
462eb30
Merge pull request #956 from mediathekview/feature/955
pidoubleyou Jan 7, 2024
5921d9e
use new experimental topics-urls
pidoubleyou Feb 4, 2024
e3ef3f9
Merge pull request #958 from mediathekview/feature/ard_topics
pidoubleyou Feb 7, 2024
041e336
parameterize sender in topic url
pidoubleyou Feb 7, 2024
1cfc733
Merge pull request #960 from mediathekview/feature/ard_topics
pidoubleyou Feb 7, 2024
41003af
init
codingPF Feb 17, 2024
0a316ac
moved to abstractPagedTask
codingPF Feb 17, 2024
8273e69
remove debug src
codingPF Feb 17, 2024
7e6ef2c
proper topic for history episodes
codingPF Feb 17, 2024
6c1fe53
cleanup and subtitles if embedded
codingPF Feb 18, 2024
c4c67ad
comment
codingPF Feb 18, 2024
d231fb6
sonar
codingPF Feb 18, 2024
cb98d9a
#965 filter ard topics
pidoubleyou Feb 19, 2024
0751f40
resolve missing subtitles
codingPF Feb 24, 2024
2e58fbf
fix for archive topic
codingPF Feb 24, 2024
bd3fd54
junit
codingPF Feb 25, 2024
2a936d3
Sonar fixes and more logging
codingPF Feb 26, 2024
6d67553
videourls from playlist
codingPF Feb 28, 2024
7dcbcf7
resolve deadlock
codingPF Feb 28, 2024
52639aa
resolve episode as last step
codingPF Feb 28, 2024
eda6bd1
Merge film creation into episodetask and test cases
codingPF Feb 29, 2024
cf3ad05
remove test case
codingPF Feb 29, 2024
381e7aa
fallback to low resolution url in case of several playlist entries
codingPF Mar 20, 2024
af81acf
remove import
codingPF Mar 20, 2024
5dee698
configure topics search
pidoubleyou Mar 23, 2024
f614070
optimize orf urls if only one stream is found
pidoubleyou Mar 23, 2024
20f8137
fix 973 (orf dummy urls)
codingPF Apr 12, 2024
9630cb8
Merge pull request #974 from mediathekview/devOrfOn
codingPF Apr 17, 2024
a17470a
add episode to title
codingPF Apr 21, 2024
b8df846
unused
codingPF Apr 25, 2024
28dc00a
ard v6
codingPF Apr 28, 2024
6198c0d
fix subtitle
codingPF Apr 28, 2024
3041195
remove system.out
pidoubleyou Apr 29, 2024
fa8a8df
add test for episodes in title
pidoubleyou Apr 29, 2024
ac39221
use two digits for season and episode in title
pidoubleyou Apr 29, 2024
e6624e1
use two digits for season and episode in title
pidoubleyou Apr 29, 2024
c71545b
add sub only if it's not empty
pidoubleyou Apr 29, 2024
1daa35a
fix to stop loading page 1 if it does not exists
codingPF May 2, 2024
b021995
add resolution
codingPF May 2, 2024
807fc64
use String.format
codingPF May 2, 2024
4ab49ce
little helper to create/validate testcases
codingPF May 2, 2024
6f6b30f
new handling of AD, DGS movies
codingPF May 2, 2024
15aad3e
add missing protocoll
codingPF May 2, 2024
74baba9
sonar
codingPF May 3, 2024
67be16c
more sonar
codingPF May 3, 2024
aeb33aa
partner filter
codingPF May 8, 2024
4d02c81
add parsevideofromthumbnail
codingPF May 8, 2024
fc4bf1f
ignore certain partners
codingPF May 9, 2024
1ba399b
update test case
codingPF May 9, 2024
41af0de
Merge pull request #976 from mediathekview/feature/942
codingPF May 11, 2024
3514aed
Merge pull request #983 from mediathekview/hotfix/981
codingPF May 11, 2024
94be6d1
clean title for UT,AD
codingPF May 17, 2024
f776edd
fix test for new AD,DGS title
codingPF May 18, 2024
547e423
Merge pull request #979 from mediathekview/hotfix/devArdv6
codingPF May 18, 2024
e846616
fix 992-01-990-989 and remove java preview feature
codingPF May 24, 2024
e2e4203
update test cases
codingPF May 24, 2024
e0c5f12
Merge pull request #994 from mediathekview/bugfix/992-991-990-989
codingPF May 25, 2024
e057760
add film even if they only have one (small) url
codingPF May 28, 2024
9f0487d
ZDF Film to Partner logic
codingPF May 31, 2024
b4360ef
remove debug source
codingPF Jun 2, 2024
bd3cc4f
enable more debug info
codingPF Jun 3, 2024
e46f16f
align unit test to optional
codingPF Jun 7, 2024
d85c395
Merge pull request #997 from mediathekview/bugfix/zdfFilm2Partner
codingPF Jun 7, 2024
40d77ba
ard: new day page url
pidoubleyou Oct 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
src/test/resources/* linguist-vendored
52 changes: 52 additions & 0 deletions .github/workflows/develop-pre-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: Develop Pre-Release
on:
push:
branches:
- "develop"

jobs:
develop-pre-release:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v1
with:
java-version: 17
java-package: jdk+fx
- name: Cache local Maven repository
uses: actions/cache@v2
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- name: Build and test
run: ./mvnw -B package docker:build -P docker

- name: SonarCloud Scan
run: ./mvnw -B org.jacoco:jacoco-maven-plugin:prepare-agent sonar:sonar -Dsonar.projectKey=mediathekview_MServer -Dsonar.organization=mediathekview -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=$SONAR_TOKEN
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

- name: Publish docker container
run: ./mvnw -B docker:push -P docker -DskipTests -Ddocker.username=$DOCKER_USERNAME -Ddocker.password=$DOCKER_PASSWORD
env:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}

- name: Publish GitHub pre-Release
uses: marvinpinto/action-automatic-releases@latest
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
automatic_release_tag: "latest-develop"
prerelease: true
title: "Development Build"
files: |
LICENSE.md
target/*.jar
target/*.tar.*
target/*.zip
target/classes/THIRD-PARTY.txt
30 changes: 30 additions & 0 deletions .github/workflows/maven.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Build and test develop

on: [ push,pull_request ]

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up JDK 17
uses: actions/setup-java@v1
with:
java-version: 17
java-package: jdk+fx
- name: Cache local Maven repository
uses: actions/cache@v2
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- name: Build and test with Maven
run: ./mvnw -B package
- name: SonarCloud Scan
run: ./mvnw -B org.jacoco:jacoco-maven-plugin:prepare-agent sonar:sonar -Dsonar.projectKey=mediathekview_MServer -Dsonar.organization=mediathekview -Dsonar.host.url=https://sonarcloud.io -Dsonar.login=$SONAR_TOKEN
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
10 changes: 8 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,16 @@ start*.sh
.classpath
.project
.settings/

*.iml
.idea
out/
build/

/src/main/resources/version.properties
version.properties
RepoZugang.properties

/logs/

logs/
/target/
/.checkstyle
Binary file added .mvn/wrapper/maven-wrapper.jar
Binary file not shown.
18 changes: 18 additions & 0 deletions .mvn/wrapper/maven-wrapper.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.6.3/apache-maven-3.6.3-bin.zip
wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.1/maven-wrapper-3.1.1.jar
49 changes: 0 additions & 49 deletions .travis.yml

This file was deleted.

97 changes: 97 additions & 0 deletions ARCHITECTURE_OVERVIEW.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
= Architecture Overview
:toc: macro

MServer is a free open-source software that crawls the media libraries of the German-speaking public broadcasters and prepares them as a machine-readable list for clients such as https://github.com/mediathekview/MediathekView[MediathekView].

toc::[]

== Context

image:docs/architecture/images/System_Context.svg[System context]

A simplified overview of the MediathekView system context.

=== Administrator (user)

The only real user of MServer is the administrator who configures and runs it.

=== Public broadcaster media libraries (external system)

Each public broadcaster media library is an own external system and needs its own crawler.
For some media libraries, no API exists, in which case the website itself is examined.

=== MediathekView (external system)

https://github.com/mediathekview/MediathekView[MediathekView] is the main supported client in the form of a desktop application.

=== MediathekViewWeb (external system)

https://github.com/mediathekview/MediathekViewWeb[MediathekViewWeb] is the second most important client in the form of a web application.

=== Other clients (external system)

Other clients like https://github.com/mediathekview/zapp[Zapp]
, https://github.com/mediathekview/plugin.video.mediathekview[MediathekView Kodi Plugin] and other, often unknown, clients.

== Quality Goals

[cols="<,>"]
|===
|Quality Goal |Motivation/description

|Easy to configure and operate (Configurability) |For the administrator, it is very easy and quick to configure and operate MServer
|Good performance (Efficiency) |The application runs very fast and uses the system resources as good as possible
|Machine-readable output (Interoperability) |Results are machine-readable and easy for clients to consume
|Good maintainability |It's easy to change and adapt to the changes in the media libraries
|Good reliability |It doesn't crash despite errors, but continues to run until its task is completed
|===

== Solution Strategy

[cols="<,>"]
|===
|Quality Goal |Matching approaches in the solution

|Easy to configure and operate (Configurability) a|
* A YAML configuration file for the entire application with overridable default values and comments in the configuration
* Packaged as executable JAR and Docker container
|Good performance (Efficiency) |Using the Java Fork/Join framework to run all crawlers at the same time and divide up their work
|Machine-readable output (Interoperability) |The results can be saved as a normal JSON file or as the custom JSON-like legacy format used by the old server
|Good maintainability a|
* Each media library has its own crawler which is stored in its own package
* The Java version and dependencies are kept up to date
* Clean code is emphasized and checked via SonarQube
* Every change is reviewed via a code review

|Good reliability |All exceptions are handled and none are passed through to the outside
|===

== Architectural Overview Diagram

image:docs/architecture/images/Architectural_Overview.svg[Architectural Overview]

An informal architecture overview diagram for MServer

=== YAML Configuration

How many threads are allowed to use, which crawlers should run and crawler configurations are set in the YAML configuration of MServer.

=== Yacl4J

A library to read the YAML configuration as objects.

=== Crawler Manager

Starts all configured crawlers, collects the results and start the filmlist writing.

=== AbstractCrawler

Contains basic logic needed in every crawler like the work splitting.

=== MLib

An internal library which contains the model objects of MediathekView.

=== Crawler

The heart of MServer crawlers for the public broadcaster media libraries.
70 changes: 70 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Code of Conduct - MediathekView

## Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to make participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.

## Our Standards

Examples of behavior that contributes to a positive environment for our
community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or
advances
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting

## Our Responsibilities

Project maintainers are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, or to ban
temporarily or permanently any contributor for other behaviors that they deem
inappropriate, threatening, offensive, or harmful.

## Scope

This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at <info@mediathekview.de>.
All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the
reporter of any incident.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant](https://contributor-covenant.org/), version
[1.4](https://www.contributor-covenant.org/version/1/4/code-of-conduct/code_of_conduct.md) and
[2.0](https://www.contributor-covenant.org/version/2/0/code_of_conduct/code_of_conduct.md),
and was generated by [contributing-gen](https://github.com/bttger/contributing-gen).
Loading
Loading