first commit
This commit is contained in:
commit
6cdf4d5197
12
.drone.yml
Normal file
12
.drone.yml
Normal file
@ -0,0 +1,12 @@
|
||||
kind: pipeline
|
||||
type: exec
|
||||
name: default
|
||||
|
||||
platform:
|
||||
os: linux
|
||||
arch: amd64
|
||||
|
||||
steps:
|
||||
- name: run
|
||||
commands:
|
||||
- docker-compose up -d --remove-orphans --build
|
3
.gitignore
vendored
Normal file
3
.gitignore
vendored
Normal file
@ -0,0 +1,3 @@
|
||||
__pycache__/
|
||||
.vscode/
|
||||
TODO.md
|
11
Dockerfile
Normal file
11
Dockerfile
Normal file
@ -0,0 +1,11 @@
|
||||
FROM tiangolo/meinheld-gunicorn-flask:python3.7
|
||||
|
||||
RUN pip install flask
|
||||
|
||||
COPY ./requirements.txt /app/requirements.txt
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN pip install -r requirements.txt
|
||||
|
||||
COPY . /app
|
625
LICENSE
Normal file
625
LICENSE
Normal file
@ -0,0 +1,625 @@
|
||||
GNU GENERAL PUBLIC LICENSE
|
||||
|
||||
Version 3, 29 June 2007
|
||||
|
||||
Copyright © 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
|
||||
Everyone is permitted to copy and distribute verbatim copies of this license
|
||||
document, but changing it is not allowed.
|
||||
|
||||
Preamble
|
||||
|
||||
The GNU General Public License is a free, copyleft license for software and
|
||||
other kinds of works.
|
||||
|
||||
The licenses for most software and other practical works are designed to take
|
||||
away your freedom to share and change the works. By contrast, the GNU General
|
||||
Public License is intended to guarantee your freedom to share and change all
|
||||
versions of a program--to make sure it remains free software for all its users.
|
||||
We, the Free Software Foundation, use the GNU General Public License for most
|
||||
of our software; it applies also to any other work released this way by its
|
||||
authors. You can apply it to your programs, too.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not price. Our
|
||||
General Public Licenses are designed to make sure that you have the freedom
|
||||
to distribute copies of free software (and charge for them if you wish), that
|
||||
you receive source code or can get it if you want it, that you can change
|
||||
the software or use pieces of it in new free programs, and that you know you
|
||||
can do these things.
|
||||
|
||||
To protect your rights, we need to prevent others from denying you these rights
|
||||
or asking you to surrender the rights. Therefore, you have certain responsibilities
|
||||
if you distribute copies of the software, or if you modify it: responsibilities
|
||||
to respect the freedom of others.
|
||||
|
||||
For example, if you distribute copies of such a program, whether gratis or
|
||||
for a fee, you must pass on to the recipients the same freedoms that you received.
|
||||
You must make sure that they, too, receive or can get the source code. And
|
||||
you must show them these terms so they know their rights.
|
||||
|
||||
Developers that use the GNU GPL protect your rights with two steps: (1) assert
|
||||
copyright on the software, and (2) offer you this License giving you legal
|
||||
permission to copy, distribute and/or modify it.
|
||||
|
||||
For the developers' and authors' protection, the GPL clearly explains that
|
||||
there is no warranty for this free software. For both users' and authors'
|
||||
sake, the GPL requires that modified versions be marked as changed, so that
|
||||
their problems will not be attributed erroneously to authors of previous versions.
|
||||
|
||||
Some devices are designed to deny users access to install or run modified
|
||||
versions of the software inside them, although the manufacturer can do so.
|
||||
This is fundamentally incompatible with the aim of protecting users' freedom
|
||||
to change the software. The systematic pattern of such abuse occurs in the
|
||||
area of products for individuals to use, which is precisely where it is most
|
||||
unacceptable. Therefore, we have designed this version of the GPL to prohibit
|
||||
the practice for those products. If such problems arise substantially in other
|
||||
domains, we stand ready to extend this provision to those domains in future
|
||||
versions of the GPL, as needed to protect the freedom of users.
|
||||
|
||||
Finally, every program is threatened constantly by software patents. States
|
||||
should not allow patents to restrict development and use of software on general-purpose
|
||||
computers, but in those that do, we wish to avoid the special danger that
|
||||
patents applied to a free program could make it effectively proprietary. To
|
||||
prevent this, the GPL assures that patents cannot be used to render the program
|
||||
non-free.
|
||||
|
||||
The precise terms and conditions for copying, distribution and modification
|
||||
follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
"This License" refers to version 3 of the GNU General Public License.
|
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of works,
|
||||
such as semiconductor masks.
|
||||
|
||||
"The Program" refers to any copyrightable work licensed under this License.
|
||||
Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals
|
||||
or organizations.
|
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work in
|
||||
a fashion requiring copyright permission, other than the making of an exact
|
||||
copy. The resulting work is called a "modified version" of the earlier work
|
||||
or a work "based on" the earlier work.
|
||||
|
||||
A "covered work" means either the unmodified Program or a work based on the
|
||||
Program.
|
||||
|
||||
To "propagate" a work means to do anything with it that, without permission,
|
||||
would make you directly or secondarily liable for infringement under applicable
|
||||
copyright law, except executing it on a computer or modifying a private copy.
|
||||
Propagation includes copying, distribution (with or without modification),
|
||||
making available to the public, and in some countries other activities as
|
||||
well.
|
||||
|
||||
To "convey" a work means any kind of propagation that enables other parties
|
||||
to make or receive copies. Mere interaction with a user through a computer
|
||||
network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices" to the
|
||||
extent that it includes a convenient and prominently visible feature that
|
||||
(1) displays an appropriate copyright notice, and (2) tells the user that
|
||||
there is no warranty for the work (except to the extent that warranties are
|
||||
provided), that licensees may convey the work under this License, and how
|
||||
to view a copy of this License. If the interface presents a list of user commands
|
||||
or options, such as a menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
|
||||
The "source code" for a work means the preferred form of the work for making
|
||||
modifications to it. "Object code" means any non-source form of a work.
|
||||
|
||||
A "Standard Interface" means an interface that either is an official standard
|
||||
defined by a recognized standards body, or, in the case of interfaces specified
|
||||
for a particular programming language, one that is widely used among developers
|
||||
working in that language.
|
||||
|
||||
The "System Libraries" of an executable work include anything, other than
|
||||
the work as a whole, that (a) is included in the normal form of packaging
|
||||
a Major Component, but which is not part of that Major Component, and (b)
|
||||
serves only to enable use of the work with that Major Component, or to implement
|
||||
a Standard Interface for which an implementation is available to the public
|
||||
in source code form. A "Major Component", in this context, means a major essential
|
||||
component (kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to produce
|
||||
the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all the source
|
||||
code needed to generate, install, and (for an executable work) run the object
|
||||
code and to modify the work, including scripts to control those activities.
|
||||
However, it does not include the work's System Libraries, or general-purpose
|
||||
tools or generally available free programs which are used unmodified in performing
|
||||
those activities but which are not part of the work. For example, Corresponding
|
||||
Source includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically linked
|
||||
subprograms that the work is specifically designed to require, such as by
|
||||
intimate data communication or control flow between those subprograms and
|
||||
other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users can regenerate
|
||||
automatically from other parts of the Corresponding Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
|
||||
All rights granted under this License are granted for the term of copyright
|
||||
on the Program, and are irrevocable provided the stated conditions are met.
|
||||
This License explicitly affirms your unlimited permission to run the unmodified
|
||||
Program. The output from running a covered work is covered by this License
|
||||
only if the output, given its content, constitutes a covered work. This License
|
||||
acknowledges your rights of fair use or other equivalent, as provided by copyright
|
||||
law.
|
||||
|
||||
You may make, run and propagate covered works that you do not convey, without
|
||||
conditions so long as your license otherwise remains in force. You may convey
|
||||
covered works to others for the sole purpose of having them make modifications
|
||||
exclusively for you, or provide you with facilities for running those works,
|
||||
provided that you comply with the terms of this License in conveying all material
|
||||
for which you do not control copyright. Those thus making or running the covered
|
||||
works for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of your copyrighted
|
||||
material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under the conditions
|
||||
stated below. Sublicensing is not allowed; section 10 makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
|
||||
No covered work shall be deemed part of an effective technological measure
|
||||
under any applicable law fulfilling obligations under article 11 of the WIPO
|
||||
copyright treaty adopted on 20 December 1996, or similar laws prohibiting
|
||||
or restricting circumvention of such measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid circumvention
|
||||
of technological measures to the extent such circumvention is effected by
|
||||
exercising rights under this License with respect to the covered work, and
|
||||
you disclaim any intention to limit operation or modification of the work
|
||||
as a means of enforcing, against the work's users, your or third parties'
|
||||
legal rights to forbid circumvention of technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
|
||||
You may convey verbatim copies of the Program's source code as you receive
|
||||
it, in any medium, provided that you conspicuously and appropriately publish
|
||||
on each copy an appropriate copyright notice; keep intact all notices stating
|
||||
that this License and any non-permissive terms added in accord with section
|
||||
7 apply to the code; keep intact all notices of the absence of any warranty;
|
||||
and give all recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey, and you
|
||||
may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
|
||||
You may convey a work based on the Program, or the modifications to produce
|
||||
it from the Program, in the form of source code under the terms of section
|
||||
4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified it, and
|
||||
giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is released under
|
||||
this License and any conditions added under section 7. This requirement modifies
|
||||
the requirement in section 4 to "keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this License to anyone
|
||||
who comes into possession of a copy. This License will therefore apply, along
|
||||
with any applicable section 7 additional terms, to the whole of the work,
|
||||
and all its parts, regardless of how they are packaged. This License gives
|
||||
no permission to license the work in any other way, but it does not invalidate
|
||||
such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display Appropriate
|
||||
Legal Notices; however, if the Program has interactive interfaces that do
|
||||
not display Appropriate Legal Notices, your work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent works,
|
||||
which are not by their nature extensions of the covered work, and which are
|
||||
not combined with it such as to form a larger program, in or on a volume of
|
||||
a storage or distribution medium, is called an "aggregate" if the compilation
|
||||
and its resulting copyright are not used to limit the access or legal rights
|
||||
of the compilation's users beyond what the individual works permit. Inclusion
|
||||
of a covered work in an aggregate does not cause this License to apply to
|
||||
the other parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
|
||||
You may convey a covered work in object code form under the terms of sections
|
||||
4 and 5, provided that you also convey the machine-readable Corresponding
|
||||
Source under the terms of this License, in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product (including
|
||||
a physical distribution medium), accompanied by the Corresponding Source fixed
|
||||
on a durable physical medium customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product (including
|
||||
a physical distribution medium), accompanied by a written offer, valid for
|
||||
at least three years and valid for as long as you offer spare parts or customer
|
||||
support for that product model, to give anyone who possesses the object code
|
||||
either (1) a copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical medium customarily
|
||||
used for software interchange, for a price no more than your reasonable cost
|
||||
of physically performing this conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the written
|
||||
offer to provide the Corresponding Source. This alternative is allowed only
|
||||
occasionally and noncommercially, and only if you received the object code
|
||||
with such an offer, in accord with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated place (gratis
|
||||
or for a charge), and offer equivalent access to the Corresponding Source
|
||||
in the same way through the same place at no further charge. You need not
|
||||
require recipients to copy the Corresponding Source along with the object
|
||||
code. If the place to copy the object code is a network server, the Corresponding
|
||||
Source may be on a different server (operated by you or a third party) that
|
||||
supports equivalent copying facilities, provided you maintain clear directions
|
||||
next to the object code saying where to find the Corresponding Source. Regardless
|
||||
of what server hosts the Corresponding Source, you remain obligated to ensure
|
||||
that it is available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided you inform
|
||||
other peers where the object code and Corresponding Source of the work are
|
||||
being offered to the general public at no charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded from
|
||||
the Corresponding Source as a System Library, need not be included in conveying
|
||||
the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any tangible
|
||||
personal property which is normally used for personal, family, or household
|
||||
purposes, or (2) anything designed or sold for incorporation into a dwelling.
|
||||
In determining whether a product is a consumer product, doubtful cases shall
|
||||
be resolved in favor of coverage. For a particular product received by a particular
|
||||
user, "normally used" refers to a typical or common use of that class of product,
|
||||
regardless of the status of the particular user or of the way in which the
|
||||
particular user actually uses, or expects or is expected to use, the product.
|
||||
A product is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent the
|
||||
only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods, procedures,
|
||||
authorization keys, or other information required to install and execute modified
|
||||
versions of a covered work in that User Product from a modified version of
|
||||
its Corresponding Source. The information must suffice to ensure that the
|
||||
continued functioning of the modified object code is in no case prevented
|
||||
or interfered with solely because modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or specifically
|
||||
for use in, a User Product, and the conveying occurs as part of a transaction
|
||||
in which the right of possession and use of the User Product is transferred
|
||||
to the recipient in perpetuity or for a fixed term (regardless of how the
|
||||
transaction is characterized), the Corresponding Source conveyed under this
|
||||
section must be accompanied by the Installation Information. But this requirement
|
||||
does not apply if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has been installed
|
||||
in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a requirement
|
||||
to continue to provide support service, warranty, or updates for a work that
|
||||
has been modified or installed by the recipient, or for the User Product in
|
||||
which it has been modified or installed. Access to a network may be denied
|
||||
when the modification itself materially and adversely affects the operation
|
||||
of the network or violates the rules and protocols for communication across
|
||||
the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided, in accord
|
||||
with this section must be in a format that is publicly documented (and with
|
||||
an implementation available to the public in source code form), and must require
|
||||
no special password or key for unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
|
||||
"Additional permissions" are terms that supplement the terms of this License
|
||||
by making exceptions from one or more of its conditions. Additional permissions
|
||||
that are applicable to the entire Program shall be treated as though they
|
||||
were included in this License, to the extent that they are valid under applicable
|
||||
law. If additional permissions apply only to part of the Program, that part
|
||||
may be used separately under those permissions, but the entire Program remains
|
||||
governed by this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option remove any
|
||||
additional permissions from that copy, or from any part of it. (Additional
|
||||
permissions may be written to require their own removal in certain cases when
|
||||
you modify the work.) You may place additional permissions on material, added
|
||||
by you to a covered work, for which you have or can give appropriate copyright
|
||||
permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you add
|
||||
to a covered work, you may (if authorized by the copyright holders of that
|
||||
material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the terms of
|
||||
sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or author
|
||||
attributions in that material or in the Appropriate Legal Notices displayed
|
||||
by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or requiring
|
||||
that modified versions of such material be marked in reasonable ways as different
|
||||
from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or authors
|
||||
of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some trade names,
|
||||
trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that material by
|
||||
anyone who conveys the material (or modified versions of it) with contractual
|
||||
assumptions of liability to the recipient, for any liability that these contractual
|
||||
assumptions directly impose on those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further restrictions"
|
||||
within the meaning of section 10. If the Program as you received it, or any
|
||||
part of it, contains a notice stating that it is governed by this License
|
||||
along with a term that is a further restriction, you may remove that term.
|
||||
If a license document contains a further restriction but permits relicensing
|
||||
or conveying under this License, you may add to a covered work material governed
|
||||
by the terms of that license document, provided that the further restriction
|
||||
does not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you must place,
|
||||
in the relevant source files, a statement of the additional terms that apply
|
||||
to those files, or a notice indicating where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the form
|
||||
of a separately written license, or stated as exceptions; the above requirements
|
||||
apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly provided
|
||||
under this License. Any attempt otherwise to propagate or modify it is void,
|
||||
and will automatically terminate your rights under this License (including
|
||||
any patent licenses granted under the third paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your license from
|
||||
a particular copyright holder is reinstated (a) provisionally, unless and
|
||||
until the copyright holder explicitly and finally terminates your license,
|
||||
and (b) permanently, if the copyright holder fails to notify you of the violation
|
||||
by some reasonable means prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is reinstated permanently
|
||||
if the copyright holder notifies you of the violation by some reasonable means,
|
||||
this is the first time you have received notice of violation of this License
|
||||
(for any work) from that copyright holder, and you cure the violation prior
|
||||
to 30 days after your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the licenses
|
||||
of parties who have received copies or rights from you under this License.
|
||||
If your rights have been terminated and not permanently reinstated, you do
|
||||
not qualify to receive new licenses for the same material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or run a copy
|
||||
of the Program. Ancillary propagation of a covered work occurring solely as
|
||||
a consequence of using peer-to-peer transmission to receive a copy likewise
|
||||
does not require acceptance. However, nothing other than this License grants
|
||||
you permission to propagate or modify any covered work. These actions infringe
|
||||
copyright if you do not accept this License. Therefore, by modifying or propagating
|
||||
a covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically receives
|
||||
a license from the original licensors, to run, modify and propagate that work,
|
||||
subject to this License. You are not responsible for enforcing compliance
|
||||
by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an organization,
|
||||
or substantially all assets of one, or subdividing an organization, or merging
|
||||
organizations. If propagation of a covered work results from an entity transaction,
|
||||
each party to that transaction who receives a copy of the work also receives
|
||||
whatever licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the Corresponding
|
||||
Source of the work from the predecessor in interest, if the predecessor has
|
||||
it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the rights
|
||||
granted or affirmed under this License. For example, you may not impose a
|
||||
license fee, royalty, or other charge for exercise of rights granted under
|
||||
this License, and you may not initiate litigation (including a cross-claim
|
||||
or counterclaim in a lawsuit) alleging that any patent claim is infringed
|
||||
by making, using, selling, offering for sale, or importing the Program or
|
||||
any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this License
|
||||
of the Program or a work on which the Program is based. The work thus licensed
|
||||
is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims owned or controlled
|
||||
by the contributor, whether already acquired or hereafter acquired, that would
|
||||
be infringed by some manner, permitted by this License, of making, using,
|
||||
or selling its contributor version, but do not include claims that would be
|
||||
infringed only as a consequence of further modification of the contributor
|
||||
version. For purposes of this definition, "control" includes the right to
|
||||
grant patent sublicenses in a manner consistent with the requirements of this
|
||||
License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free patent
|
||||
license under the contributor's essential patent claims, to make, use, sell,
|
||||
offer for sale, import and otherwise run, modify and propagate the contents
|
||||
of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express agreement
|
||||
or commitment, however denominated, not to enforce a patent (such as an express
|
||||
permission to practice a patent or covenant not to sue for patent infringement).
|
||||
To "grant" such a patent license to a party means to make such an agreement
|
||||
or commitment not to enforce a patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license, and the
|
||||
Corresponding Source of the work is not available for anyone to copy, free
|
||||
of charge and under the terms of this License, through a publicly available
|
||||
network server or other readily accessible means, then you must either (1)
|
||||
cause the Corresponding Source to be so available, or (2) arrange to deprive
|
||||
yourself of the benefit of the patent license for this particular work, or
|
||||
(3) arrange, in a manner consistent with the requirements of this License,
|
||||
to extend the patent license to downstream recipients. "Knowingly relying"
|
||||
means you have actual knowledge that, but for the patent license, your conveying
|
||||
the covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that country
|
||||
that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or arrangement,
|
||||
you convey, or propagate by procuring conveyance of, a covered work, and grant
|
||||
a patent license to some of the parties receiving the covered work authorizing
|
||||
them to use, propagate, modify or convey a specific copy of the covered work,
|
||||
then the patent license you grant is automatically extended to all recipients
|
||||
of the covered work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within the scope
|
||||
of its coverage, prohibits the exercise of, or is conditioned on the non-exercise
|
||||
of one or more of the rights that are specifically granted under this License.
|
||||
You may not convey a covered work if you are a party to an arrangement with
|
||||
a third party that is in the business of distributing software, under which
|
||||
you make payment to the third party based on the extent of your activity of
|
||||
conveying the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory patent
|
||||
license (a) in connection with copies of the covered work conveyed by you
|
||||
(or copies made from those copies), or (b) primarily for and in connection
|
||||
with specific products or compilations that contain the covered work, unless
|
||||
you entered into that arrangement, or that patent license was granted, prior
|
||||
to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting any implied
|
||||
license or other defenses to infringement that may otherwise be available
|
||||
to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or otherwise)
|
||||
that contradict the conditions of this License, they do not excuse you from
|
||||
the conditions of this License. If you cannot convey a covered work so as
|
||||
to satisfy simultaneously your obligations under this License and any other
|
||||
pertinent obligations, then as a consequence you may not convey it at all.
|
||||
For example, if you agree to terms that obligate you to collect a royalty
|
||||
for further conveying from those to whom you convey the Program, the only
|
||||
way you could satisfy both those terms and this License would be to refrain
|
||||
entirely from conveying the Program.
|
||||
|
||||
13. Use with the GNU Affero General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, you have permission to
|
||||
link or combine any covered work with a work licensed under version 3 of the
|
||||
GNU Affero General Public License into a single combined work, and to convey
|
||||
the resulting work. The terms of this License will continue to apply to the
|
||||
part which is the covered work, but the special requirements of the GNU Affero
|
||||
General Public License, section 13, concerning interaction through a network
|
||||
will apply to the combination as such.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of the
|
||||
GNU General Public License from time to time. Such new versions will be similar
|
||||
in spirit to the present version, but may differ in detail to address new
|
||||
problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the Program specifies
|
||||
that a certain numbered version of the GNU General Public License "or any
|
||||
later version" applies to it, you have the option of following the terms and
|
||||
conditions either of that numbered version or of any later version published
|
||||
by the Free Software Foundation. If the Program does not specify a version
|
||||
number of the GNU General Public License, you may choose any version ever
|
||||
published by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future versions of
|
||||
the GNU General Public License can be used, that proxy's public statement
|
||||
of acceptance of a version permanently authorizes you to choose that version
|
||||
for the Program.
|
||||
|
||||
Later license versions may give you additional or different permissions. However,
|
||||
no additional obligations are imposed on any author or copyright holder as
|
||||
a result of your choosing to follow a later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE
|
||||
LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
|
||||
OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER
|
||||
EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
|
||||
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
|
||||
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM
|
||||
PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR
|
||||
CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL
|
||||
ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM
|
||||
AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL,
|
||||
INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO
|
||||
USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED
|
||||
INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE
|
||||
PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER
|
||||
PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided above cannot
|
||||
be given local legal effect according to their terms, reviewing courts shall
|
||||
apply local law that most closely approximates an absolute waiver of all civil
|
||||
liability in connection with the Program, unless a warranty or assumption
|
||||
of liability accompanies a copy of the Program in return for a fee. END OF
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest possible
|
||||
use to the public, the best way to achieve this is to make it free software
|
||||
which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest to attach
|
||||
them to the start of each source file to most effectively state the exclusion
|
||||
of warranty; and each file should have at least the "copyright" line and a
|
||||
pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify it under
|
||||
the terms of the GNU General Public License as published by the Free Software
|
||||
Foundation, either version 3 of the License, or (at your option) any later
|
||||
version.
|
||||
|
||||
This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
||||
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License along with
|
||||
this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If the program does terminal interaction, make it output a short notice like
|
||||
this when it starts in an interactive mode:
|
||||
|
||||
<program> Copyright (C) <year> <name of author>
|
||||
|
||||
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||
|
||||
This is free software, and you are welcome to redistribute it under certain
|
||||
conditions; type `show c' for details.
|
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||
parts of the General Public License. Of course, your program's commands might
|
||||
be different; for a GUI interface, you would use an "about box".
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a "copyright disclaimer" for the program, if necessary. For
|
||||
more information on this, and how to apply and follow the GNU GPL, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
The GNU General Public License does not permit incorporating your program
|
||||
into proprietary programs. If your program is a subroutine library, you may
|
||||
consider it more useful to permit linking proprietary applications with the
|
||||
library. If this is what you want to do, use the GNU Lesser General Public
|
||||
License instead of this License. But first, please read <https://www.gnu.org/
|
||||
licenses /why-not-lgpl.html>.
|
5
README.md
Normal file
5
README.md
Normal file
@ -0,0 +1,5 @@
|
||||
# sdc-lifecycle
|
||||
|
||||
[![Build Status](https://drone.med.upct.es/api/badges/ocean/sdc-lifecycle/status.svg)](https://drone.med.upct.es/ocean/sdc-lifecycle)
|
||||
|
||||
Web tool to upload, process and store data collected during the campaigns
|
4
dataset/.gitignore
vendored
Normal file
4
dataset/.gitignore
vendored
Normal file
@ -0,0 +1,4 @@
|
||||
*
|
||||
# Except this files
|
||||
!.gitignore
|
||||
!README.md
|
2
dataset/README.md
Normal file
2
dataset/README.md
Normal file
@ -0,0 +1,2 @@
|
||||
This folder contains all the sensor data uploaded to the server
|
||||
|
27
docker-compose.yml
Normal file
27
docker-compose.yml
Normal file
@ -0,0 +1,27 @@
|
||||
version: '3.7'
|
||||
|
||||
services:
|
||||
|
||||
## TODO: Add true proxy WSGI Server instead of running the builtin
|
||||
uploadtool:
|
||||
build: .
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /mnt/storage/dataset-neo:/app/dataset
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
- traefik.http.routers.sdc-uploader.entryPoints=web-secure
|
||||
- traefik.http.routers.sdc-uploader.rule=Host(`upload.med.upct.es`)
|
||||
- traefik.http.routers.sdc-uploader.tls.certresolver=default
|
||||
|
||||
download:
|
||||
image: abiosoft/caddy
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- /mnt/storage/dataset-neo:/srv
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
- traefik.http.routers.sdc-download.entryPoints=web-secure
|
||||
- traefik.http.routers.sdc-download.rule=Host(`download.med.upct.es`)
|
||||
- traefik.http.routers.sdc-download.tls.certresolver=default
|
||||
- traefik.http.services.sdc-download.loadbalancer.server.port=2015
|
46
gunicorn_conf.py
Normal file
46
gunicorn_conf.py
Normal file
@ -0,0 +1,46 @@
|
||||
# cat /gunicorn_conf.py inside container
|
||||
# Copied to project so we can tune max_content_length
|
||||
from __future__ import print_function
|
||||
|
||||
import json
|
||||
import multiprocessing
|
||||
import os
|
||||
|
||||
workers_per_core_str = os.getenv("WORKERS_PER_CORE", "2")
|
||||
web_concurrency_str = os.getenv("WEB_CONCURRENCY", None)
|
||||
host = os.getenv("HOST", "0.0.0.0")
|
||||
port = os.getenv("PORT", "80")
|
||||
bind_env = os.getenv("BIND", None)
|
||||
use_loglevel = os.getenv("LOG_LEVEL", "info")
|
||||
if bind_env:
|
||||
use_bind = bind_env
|
||||
else:
|
||||
use_bind = "{host}:{port}".format(host=host, port=port)
|
||||
|
||||
cores = multiprocessing.cpu_count()
|
||||
workers_per_core = float(workers_per_core_str)
|
||||
default_web_concurrency = workers_per_core * cores
|
||||
if web_concurrency_str:
|
||||
web_concurrency = int(web_concurrency_str)
|
||||
assert web_concurrency > 0
|
||||
else:
|
||||
web_concurrency = int(default_web_concurrency)
|
||||
|
||||
# Gunicorn config variables
|
||||
loglevel = use_loglevel
|
||||
workers = web_concurrency
|
||||
bind = use_bind
|
||||
keepalive = 120
|
||||
errorlog = "-"
|
||||
|
||||
# For debugging and testing
|
||||
log_data = {
|
||||
"loglevel": loglevel,
|
||||
"workers": workers,
|
||||
"bind": bind,
|
||||
# Additional, non-gunicorn variables
|
||||
"workers_per_core": workers_per_core,
|
||||
"host": host,
|
||||
"port": port,
|
||||
}
|
||||
print(json.dumps(log_data))
|
196
main.py
Normal file
196
main.py
Normal file
@ -0,0 +1,196 @@
|
||||
from flask import Flask, render_template, request
|
||||
from datetime import datetime
|
||||
from probeutils import utils as probeutils
|
||||
import re
|
||||
import fnmatch
|
||||
import os
|
||||
import meinheld
|
||||
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
app.config['UPLOAD_FOLDER'] = "./dataset/"
|
||||
|
||||
app.config['MAX_CONTENT_LENGTH'] = 10000000000 # 10GB
|
||||
meinheld.set_max_content_length(100*1024*1024)
|
||||
|
||||
app.config['DOWNLOADS_URL'] = "https://download.med.upct.es/"
|
||||
|
||||
probes = [{
|
||||
'sensor': 'SUNA',
|
||||
'img': 'suna.jpg',
|
||||
'active': ['SATSLF0037'],
|
||||
'stations': ['M1', 'M2', 'M3']
|
||||
}, {
|
||||
'sensor': 'FIRe',
|
||||
'img': 'fire.jpg',
|
||||
'active': ['SATFIS0006'],
|
||||
'stations': ['M1', 'M2', 'M3']
|
||||
}, {
|
||||
'sensor': 'PhycoCTD',
|
||||
'img': 'phycoctd.jpg',
|
||||
'active': ['phyco_v1', 'phyco_v2'],
|
||||
'stations': ['M1', 'M2', 'M3']
|
||||
}, {
|
||||
'sensor': 'CastAway',
|
||||
'img': 'castaway.jpg',
|
||||
'active': ['CC1326008'],
|
||||
'stations': ['M1', 'M2', 'M3']
|
||||
}
|
||||
#,{
|
||||
# 'sensor': 'GoPro',
|
||||
# 'img': 'gopro.png',
|
||||
# 'active': ['gopro1'],
|
||||
# 'stations': ['M1', 'M2', 'M3']
|
||||
#}
|
||||
]
|
||||
|
||||
# Return our beautiful Bootstrap webpage. That we totally have.
|
||||
@app.route('/')
|
||||
def upload():
|
||||
return render_template('upload.html', probes=probes)
|
||||
|
||||
# What happens when the files just don't fit
|
||||
@app.errorhandler(413)
|
||||
def request_entity_too_large(error):
|
||||
return 'File Too Thicc', 413
|
||||
|
||||
# What happens when we did an oopsie
|
||||
@app.errorhandler(500)
|
||||
def internal_server_error(error):
|
||||
return render_template('error.html')
|
||||
|
||||
# What happens when someone uploads a thingy
|
||||
@app.route('/uploader', methods=['POST'])
|
||||
def upload_file():
|
||||
if request.method == 'POST':
|
||||
f = request.files['file']
|
||||
|
||||
# Get which probe the user uploaded
|
||||
probe = request.form.get('probe')
|
||||
|
||||
# Get selected probe
|
||||
activeProbe = request.form.get('activeProbe')
|
||||
|
||||
# Check if its a valid file to upload
|
||||
# TODO: Implement this method
|
||||
# probeutils.check(probe, f)
|
||||
|
||||
# Check if folder exists
|
||||
try:
|
||||
if not os.path.exists(
|
||||
os.path.join(app.config['UPLOAD_FOLDER'], probe)):
|
||||
os.makedirs(os.path.join(app.config['UPLOAD_FOLDER'], probe))
|
||||
|
||||
if not os.path.exists(
|
||||
os.path.join(app.config['UPLOAD_FOLDER'], probe, 'raw')):
|
||||
os.makedirs(os.path.join(
|
||||
app.config['UPLOAD_FOLDER'], probe, 'raw'))
|
||||
except Exception:
|
||||
return render_template('error.html')
|
||||
|
||||
# Strip station
|
||||
# if forceStation is checked, override all station parsing
|
||||
if (request.form.get("forceStation" + probe) != None):
|
||||
station = request.form.get("stations" + probe)
|
||||
else:
|
||||
if (fnmatch.fnmatch((f.filename).upper(), '*M1*') or fnmatch.fnmatch((f.filename).upper(), 'M1*')):
|
||||
station = 'M1'
|
||||
elif (fnmatch.fnmatch((f.filename).upper(), "*M2*") or fnmatch.fnmatch((f.filename).upper(), "M2*")):
|
||||
station = 'M2'
|
||||
elif (fnmatch.fnmatch((f.filename).upper(), "*M3*") or fnmatch.fnmatch((f.filename).upper(), "M3*")):
|
||||
station = 'M3'
|
||||
|
||||
# Date Parser of filename
|
||||
match1 = re.search('\d{4}-\d{2}-\d{2}', f.filename)
|
||||
if (match1 == None):
|
||||
match1 = re.search('\d{2}-\d{2}-\d{4}', f.filename)
|
||||
try:
|
||||
date = datetime.strptime(match1.group(), '%d-%m-%Y').date()
|
||||
except Exception:
|
||||
# Error! More strange data format XD
|
||||
match1 = re.search('\d{4}\d{2}\d{2}', f.filename)
|
||||
if (match1 == None):
|
||||
match1 = re.search('\d{2}\d{2}\d{4}', f.filename)
|
||||
date = datetime.strptime(match1.group(), '%d%m%Y').date()
|
||||
else:
|
||||
date = datetime.strptime(match1.group(), '%Y%m%d').date()
|
||||
else:
|
||||
date = datetime.strptime(match1.group(), '%Y-%m-%d').date()
|
||||
|
||||
# Crafting the real overpowered name!!!!
|
||||
filename = probe + '-' + station + '-'+ activeProbe + '-'+ date.strftime("%Y-%m-%d") + '.csv'
|
||||
#print(filename)
|
||||
|
||||
# Setting the filePath and saving it
|
||||
finalFilename = 'raw-'+filename
|
||||
rawFilePath = os.path.join(app.config['UPLOAD_FOLDER'], probe, 'raw', finalFilename)
|
||||
f.save(rawFilePath)
|
||||
|
||||
# Start with probe definition
|
||||
if (probe == 'CastAway'):
|
||||
''' CastAway Files '''
|
||||
|
||||
df_table = probeutils.process_castaway(rawFilePath)
|
||||
|
||||
elif (probe == 'SUNA'):
|
||||
''' SUNA Files '''
|
||||
|
||||
df_table = probeutils.process_suna(rawFilePath)
|
||||
|
||||
elif (probe == 'FIRe'):
|
||||
''' FIRe Files '''
|
||||
|
||||
df_table = probeutils.process_fire(rawFilePath)
|
||||
|
||||
elif (probe == 'PhycoCTD'):
|
||||
''' PhycoCTD Files '''
|
||||
|
||||
df_table = probeutils.process_phyco(rawFilePath)
|
||||
else:
|
||||
''' Empty probe '''
|
||||
|
||||
df_table = "<div></div>"
|
||||
|
||||
# Save file on raw folder and create URL download
|
||||
url_download = app.config['DOWNLOADS_URL'] + probe + "/" + finalFilename
|
||||
|
||||
# Return success webpage
|
||||
return render_template('successful.html', url=url_download, tables=df_table)
|
||||
else:
|
||||
# No POST Found
|
||||
return render_template('error.html')
|
||||
|
||||
## Force regenerate QC
|
||||
@app.route('/regenerate')
|
||||
def regenerate():
|
||||
error_list = []
|
||||
#for each file in raw folder, execute all QC
|
||||
PATH = './dataset/FIRe/raw'
|
||||
for filename in os.listdir(PATH):
|
||||
try:
|
||||
filename_nn = probeutils.execution_NN(filename)
|
||||
probeutils.execution_PAR(filename_nn)
|
||||
except Exception as err:
|
||||
#print(filename)
|
||||
#print(repr(err))
|
||||
error_list.append([filename, repr(err)])
|
||||
|
||||
#print('Regenerate FIRe done')
|
||||
|
||||
PATH = './dataset/PhycoCTD/raw'
|
||||
for filename in os.listdir(PATH):
|
||||
try:
|
||||
probeutils.check_if_old_phyco(filename)
|
||||
probeutils.execution_pb_phyco(filename)
|
||||
except Exception as err:
|
||||
#print(filename)
|
||||
#print(repr(err))
|
||||
error_list.append([filename, repr(err)])
|
||||
|
||||
#print('Regenerate Phyco done')
|
||||
|
||||
return render_template('regenerate.html', error_list=error_list)
|
||||
|
||||
if __name__ == '__main__':
|
||||
app.run()
|
293
probeutils/utils.py
Normal file
293
probeutils/utils.py
Normal file
@ -0,0 +1,293 @@
|
||||
from datetime import datetime
|
||||
from shutil import copy2
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
import os, csv
|
||||
|
||||
UPLOAD_FOLDER = './dataset/'
|
||||
tableClass = 'table table-striped table-sm table-responsive'
|
||||
|
||||
def get_date(probe, file):
|
||||
''' Test utils file '''
|
||||
return datetime.now().strftime("%Y-%m-%d-%H:%M:%S")
|
||||
|
||||
# TODO:
|
||||
# - Check if is a good file
|
||||
#
|
||||
#def checkUploadedFile(probe, file):
|
||||
# return 1
|
||||
|
||||
# TODO:
|
||||
# - clean the head of csv, need to extract info an remove the %% (maybe new file with metadata info?) [done]
|
||||
# - create the empty csv, first row with headers [done]
|
||||
# - desviation and more data things
|
||||
def process_castaway(file):
|
||||
|
||||
try:
|
||||
# Copy raw file to process folder
|
||||
with open(file) as CastAwayFile:
|
||||
filename = os.path.basename(CastAwayFile.name).strip('raw-')
|
||||
processFilePath = os.path.join(
|
||||
UPLOAD_FOLDER, 'CastAway', filename)
|
||||
#copy2(file, processFilePath)
|
||||
|
||||
# Getting some metadata info
|
||||
#with open(file, newline='') as castfile:
|
||||
# lines = castfile.readlines()
|
||||
# device = lines[0].split(',')[1].replace('\r\n', '')
|
||||
# filename = lines[1].split(',')[1].replace('\r\n', '')
|
||||
# start_latitude = lines[9].split(',')[1].replace('\r\n', '')
|
||||
# start_longitude = lines[10].split(',')[1].replace('\r\n', '')
|
||||
# start_altitude = lines[11].split(',')[1].replace('\r\n', '')
|
||||
# castfile.close()
|
||||
|
||||
# Opening the csv with pandas
|
||||
df = pd.read_csv(file, skiprows=28)
|
||||
|
||||
# Extract perfil bajada, el 1 es por el header
|
||||
index_max = df['Depth (Meter)'].idxmax() + 1
|
||||
#df_perfilBajada = df[df['depth'].between(0, df['depth'].max())] # No funciona muy bien
|
||||
# Limitamos a solo el perfil de bajada
|
||||
df_perfilBajada = df[:index_max]
|
||||
|
||||
# Guardamos solo el perfilBajada en carpeta PB
|
||||
if not os.path.exists(os.path.join(UPLOAD_FOLDER, 'CastAway', 'PB')):
|
||||
os.makedirs(os.path.join(UPLOAD_FOLDER, 'CastAway', 'PB'))
|
||||
|
||||
filenamePB = filename.strip('.csv') + '-PB.csv'
|
||||
perfilBajadaFilePath = os.path.join(
|
||||
UPLOAD_FOLDER, 'CastAway', 'PB', filenamePB)
|
||||
df_perfilBajada.to_csv(perfilBajadaFilePath, index=False)
|
||||
|
||||
# Trying to show Dataframe on webpage
|
||||
return df.to_html(classes=tableClass)
|
||||
|
||||
except Exception as ex:
|
||||
print('Exception: '+repr(ex))
|
||||
return ex
|
||||
|
||||
def process_suna(file):
|
||||
try:
|
||||
df = pd.read_csv(file, encoding="ISO-8859-1", header=None)
|
||||
|
||||
#df.columns = ['fechaHora', 'INSTRUMENT', 'Start-time', 'Nitrato(uMol/L)','Nitrato(MG/L)', 'ERROR', 'T_lamp', ]
|
||||
|
||||
#if not os.path.exists(os.path.join(UPLOAD_FOLDER, 'SUNA', 'HEAD')):
|
||||
# os.makedirs(os.path.join(UPLOAD_FOLDER, 'SUNA', 'HEAD'))
|
||||
|
||||
# Return webpage
|
||||
return df.to_html(classes=tableClass)
|
||||
except Exception as ex:
|
||||
print('Exception: '+repr(ex))
|
||||
return ex
|
||||
|
||||
# TODO:
|
||||
# - Ask for FIRe examples, actually only bin files found (done)
|
||||
# - Extract file to process folder (done)
|
||||
# - Save a File with PAR info
|
||||
# - Headers on email (ask for new headers)[done]
|
||||
# - Remove negative values (this, done)
|
||||
# - Arrange similar windows depth and do measure
|
||||
def process_fire(file):
|
||||
''' Processing FIRe '''
|
||||
|
||||
try:
|
||||
# Copy raw file to process folder
|
||||
with open(file) as FIReFile:
|
||||
filename_raw = os.path.basename(FIReFile.name)
|
||||
filename = os.path.basename(FIReFile.name).strip('raw-')
|
||||
#raw_file_path = os.path.join(UPLOAD_FOLDER, 'FIRe', 'raw', filename_raw)
|
||||
processFilePath = os.path.join(
|
||||
UPLOAD_FOLDER, 'FIRe', filename)
|
||||
#copy2(file, processFilePath)
|
||||
|
||||
# First, need to check the csv headers
|
||||
# Open the process file with pandas
|
||||
df = pd.read_csv(file, header=None)
|
||||
|
||||
# Headers (now working fine)
|
||||
df.columns = ['fechaHora', 'estacion', 'fecha', 'hora', 'profundidad', 'Fo', 'Fm', 'Fv', 'Fv/Fm', 'p', 'Abs_rel', 'Abs_abs', 'led_light', 'ETR', 'coma1', 'coma2', 'coma3', 'coma4', 'coma5', 'coma6','error_norm', 'PAR', 'V', 'cero1', 'cero2', 'cero3', 'cero4', 'raro']
|
||||
df.loc[:, 'coma1'] = 0
|
||||
df.loc[:, 'coma2'] = 0
|
||||
df.loc[:, 'coma3'] = 0
|
||||
df.loc[:, 'coma4'] = 0
|
||||
df.loc[:, 'coma5'] = 0
|
||||
df.loc[:, 'coma6'] = 0
|
||||
|
||||
# Fixing empty values
|
||||
#df.to_csv(os.path.join(UPLOAD_FOLDER, 'FIRe', 'raw', 'raw-'+filename), index=False)
|
||||
|
||||
filename_nn = execution_NN(filename_raw)
|
||||
|
||||
execution_PAR(filename_nn)
|
||||
|
||||
return df.to_html(classes=tableClass)
|
||||
|
||||
except Exception as ex:
|
||||
print('Exception: '+repr(ex))
|
||||
return ex
|
||||
|
||||
# TODO:
|
||||
# - Only perfil bajada (done)
|
||||
def process_phyco(file):
|
||||
''' Processing PhycoCTD'''
|
||||
|
||||
try:
|
||||
# Copy raw file to process folder
|
||||
with open(file, "r+") as phycoFile:
|
||||
# Sustract raw- of filename
|
||||
filename_raw = os.path.basename(phycoFile.name)
|
||||
filename = os.path.basename(phycoFile.name).strip('raw-')
|
||||
processFilePath = os.path.join(
|
||||
UPLOAD_FOLDER, 'PhycoCTD', filename)
|
||||
#copy2(file, processFilePath)
|
||||
|
||||
# Working with the process file
|
||||
df = pd.read_csv(file, delimiter=';')
|
||||
|
||||
check_if_old_phyco(filename_raw)
|
||||
execution_pb_phyco(filename_raw)
|
||||
|
||||
# Return webpage
|
||||
return df.to_html(classes=tableClass)
|
||||
except Exception as ex:
|
||||
print('Exception: '+repr(ex))
|
||||
return ex
|
||||
|
||||
def check_if_old_phyco(filename):
|
||||
# Checking if old file csv
|
||||
PATH = os.path.join(UPLOAD_FOLDER, 'PhycoCTD', 'raw', filename)
|
||||
with open(PATH, "r+") as phycoFile:
|
||||
# Check if old version
|
||||
line = phycoFile.readline()
|
||||
if (',' in line):
|
||||
old = True
|
||||
else:
|
||||
old = False
|
||||
|
||||
if (old == True):
|
||||
df_old = pd.read_csv(PATH, header=None, skiprows=1)
|
||||
if (len(df_old.columns) == 16):
|
||||
df_old.columns = ['station', 'latitude', 'longitude', 'time', 'depth', 'temp1', 'temp2', 'cdom[gain]',
|
||||
'cdom[ppb]', 'cdom[mv]', 'pe[gain]', 'pe[ppb]', 'pe[mv]', 'chl[gain]', 'chl[ppb]', 'chl[mv]']
|
||||
else:
|
||||
df_old.columns = ['station', 'time', 'depth', 'temp1', 'temp2', 'cdom[gain]', 'cdom[ppb]',
|
||||
'cdom[mv]', 'pe[gain]', 'pe[ppb]', 'pe[mv]', 'chl[gain]', 'chl[ppb]', 'chl[mv]']
|
||||
raw_path = os.path.join(UPLOAD_FOLDER, 'PhycoCTD', 'raw', filename)
|
||||
df_old.to_csv(raw_path, index=False, sep=';')
|
||||
|
||||
def execution_pb_phyco(filename):
|
||||
# Guardamos solo el perfilBajada
|
||||
if not os.path.exists(os.path.join(UPLOAD_FOLDER, 'PhycoCTD', 'PB')):
|
||||
os.makedirs(os.path.join(UPLOAD_FOLDER, 'PhycoCTD', 'PB'))
|
||||
|
||||
PATH = os.path.join(UPLOAD_FOLDER, 'PhycoCTD', 'raw', filename)
|
||||
df = pd.read_csv(PATH, delimiter=';')
|
||||
|
||||
# Checking if NaN values exists [for hack lab version csv upload]
|
||||
if (df['temp2'].isnull().sum() > 0):
|
||||
df.loc[:, 'temp2'] = 0
|
||||
df = df.dropna()
|
||||
|
||||
# Extract perfil bajada, el 1 es por el header
|
||||
index_max = df['depth'].idxmax() + 1
|
||||
#df_perfilBajada = df[df['depth'].between(0, df['depth'].max())] # No funciona muy bien
|
||||
# Limitamos a solo el perfil de bajada
|
||||
df_pb= df[:index_max]
|
||||
|
||||
filename_pb = filename.strip('raw-').strip('.csv') + '-PB.csv'
|
||||
pb_path = os.path.join(UPLOAD_FOLDER, 'PhycoCTD', 'PB', filename_pb)
|
||||
df_pb.to_csv(pb_path, sep=';', index=False)
|
||||
|
||||
def execution_NN(filename):
|
||||
|
||||
PATH = os.path.join(UPLOAD_FOLDER, 'FIRe', 'raw', filename)
|
||||
df = pd.read_csv(PATH, header=None)
|
||||
|
||||
# Headers (now working fine)
|
||||
df.columns = ['fechaHora', 'estacion', 'fecha', 'hora', 'profundidad', 'Fo', 'Fm', 'Fv', 'Fv/Fm', 'p', 'Abs_rel', 'Abs_abs', 'led_light',
|
||||
'ETR', 'coma1', 'coma2', 'coma3', 'coma4', 'coma5', 'coma6', 'error_norm', 'PAR', 'V', 'cero1', 'cero2', 'cero3', 'cero4', 'raro']
|
||||
df.loc[:, 'coma1'] = 0
|
||||
df.loc[:, 'coma2'] = 0
|
||||
df.loc[:, 'coma3'] = 0
|
||||
df.loc[:, 'coma4'] = 0
|
||||
df.loc[:, 'coma5'] = 0
|
||||
df.loc[:, 'coma6'] = 0
|
||||
|
||||
# NonNegative values rutine
|
||||
if not os.path.exists(os.path.join(UPLOAD_FOLDER, 'FIRe', 'NN')):
|
||||
os.makedirs(os.path.join(UPLOAD_FOLDER, 'FIRe', 'NN'))
|
||||
|
||||
# Remove rows with negatives values
|
||||
df_nn = df[(df.iloc[:, 4:27] >= 0).all(1)]
|
||||
filename_nn = filename.strip('raw-').strip('.csv') + '-NN.csv'
|
||||
nonnegative_path = os.path.join(UPLOAD_FOLDER, 'FIRe', 'NN', filename_nn)
|
||||
df_nn.to_csv(nonnegative_path, index=False)
|
||||
|
||||
return filename_nn
|
||||
|
||||
def execution_PAR(filename):
|
||||
# PAR rutine
|
||||
if not os.path.exists(os.path.join(UPLOAD_FOLDER, 'FIRe', 'PAR')):
|
||||
os.makedirs(os.path.join(UPLOAD_FOLDER, 'FIRe', 'PAR'))
|
||||
|
||||
PATH = os.path.join(UPLOAD_FOLDER, 'FIRe', 'NN', filename)
|
||||
df = pd.read_csv(PATH)
|
||||
|
||||
# PAR execution
|
||||
PAR_columns = ['estacion', 'fecha', 'profundidad', 'Fo', 'Fm', 'Fv', 'Fv/Fm', 'p', 'Abs_rel', 'Abs_abs', 'led_light',
|
||||
'ETR', 'error_norm', 'PAR']
|
||||
|
||||
df_PAR = pd.DataFrame(columns=PAR_columns)
|
||||
|
||||
index_max = df['profundidad'].idxmax()
|
||||
index_min = df['profundidad'].idxmin() + 1
|
||||
depth_list = df[index_max:index_min]['profundidad'].to_list()
|
||||
|
||||
last_value = df['profundidad'].max()
|
||||
similar_depth = []
|
||||
for i in range(len(depth_list)):
|
||||
#print(depth_list[i])
|
||||
|
||||
if (abs(last_value - depth_list[i]) <= 2000):
|
||||
last_value = depth_list[i]
|
||||
similar_depth.append(depth_list[i])
|
||||
elif (abs(last_value - depth_list[i]) >= 6000):
|
||||
## Save the actual list to DataFrame
|
||||
#print(similar_depth)
|
||||
index_first = pd.to_numeric(df.index[df['profundidad'] == similar_depth[0]])[0]
|
||||
index_last = pd.to_numeric(df.index[df['profundidad'] == similar_depth[-1]])[0] + 1
|
||||
df_range = df.iloc[index_first:index_last]
|
||||
|
||||
# Working with df_range
|
||||
estacion = df['estacion'][0]
|
||||
fecha = df['fecha'][0]
|
||||
profundidad = df_range['profundidad'].mean()
|
||||
Fo = df_range['Fo'].mean()
|
||||
Fm = df_range['Fm'].mean()
|
||||
Fv = df_range['Fv'].mean()
|
||||
FvFm = df_range['Fv/Fm'].mean()
|
||||
p = df_range['p'].mean()
|
||||
Abs_rel = df_range['Abs_rel'].mean()
|
||||
Abs_abs = df_range['Abs_abs'].mean()
|
||||
led_light = df_range['led_light'].mean()
|
||||
ETR = df_range['ETR'].mean()
|
||||
error_norm = df_range['error_norm'].mean()
|
||||
PAR = df_range['PAR'].mean()
|
||||
data = [estacion, fecha, profundidad, Fo, Fm, Fv, FvFm, p, Abs_rel, Abs_abs, led_light, ETR, error_norm, PAR]
|
||||
row = pd.Series(data, index=PAR_columns)
|
||||
df_PAR = df_PAR.append(row, ignore_index=True)
|
||||
|
||||
## Empty the list
|
||||
similar_depth = []
|
||||
last_value = depth_list[i]
|
||||
|
||||
## Adding to the new list
|
||||
similar_depth.append(depth_list[i])
|
||||
|
||||
# Saving the DataFrame
|
||||
# TODO: check last values, around 400 depth
|
||||
filenamePAR = filename.strip('-NN.csv') + '-PAR.csv'
|
||||
PARFilePath = os.path.join(UPLOAD_FOLDER, 'FIRe', 'PAR', filenamePAR)
|
||||
df_PAR.to_csv(PARFilePath, index=False)
|
||||
|
3
requirements.txt
Normal file
3
requirements.txt
Normal file
@ -0,0 +1,3 @@
|
||||
Flask
|
||||
pandas
|
||||
meinheld
|
265
static/css/main.css
Normal file
265
static/css/main.css
Normal file
@ -0,0 +1,265 @@
|
||||
/*! HTML5 Boilerplate v7.3.0 | MIT License | https://html5boilerplate.com/ */
|
||||
|
||||
/* main.css 2.0.0 | MIT License | https://github.com/h5bp/main.css#readme */
|
||||
/*
|
||||
* What follows is the result of much research on cross-browser styling.
|
||||
* Credit left inline and big thanks to Nicolas Gallagher, Jonathan Neal,
|
||||
* Kroc Camen, and the H5BP dev community and team.
|
||||
*/
|
||||
|
||||
/* ==========================================================================
|
||||
Base styles: opinionated defaults
|
||||
========================================================================== */
|
||||
|
||||
html {
|
||||
color: #222;
|
||||
font-size: 1em;
|
||||
line-height: 1.4;
|
||||
}
|
||||
|
||||
/*
|
||||
* Remove text-shadow in selection highlight:
|
||||
* https://twitter.com/miketaylr/status/12228805301
|
||||
*
|
||||
* Vendor-prefixed and regular ::selection selectors cannot be combined:
|
||||
* https://stackoverflow.com/a/16982510/7133471
|
||||
*
|
||||
* Customize the background color to match your design.
|
||||
*/
|
||||
|
||||
::-moz-selection {
|
||||
background: #b3d4fc;
|
||||
text-shadow: none;
|
||||
}
|
||||
|
||||
::selection {
|
||||
background: #b3d4fc;
|
||||
text-shadow: none;
|
||||
}
|
||||
|
||||
/*
|
||||
* A better looking default horizontal rule
|
||||
*/
|
||||
|
||||
hr {
|
||||
display: block;
|
||||
height: 1px;
|
||||
border: 0;
|
||||
border-top: 1px solid #ccc;
|
||||
margin: 1em 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/*
|
||||
* Remove the gap between audio, canvas, iframes,
|
||||
* images, videos and the bottom of their containers:
|
||||
* https://github.com/h5bp/html5-boilerplate/issues/440
|
||||
*/
|
||||
|
||||
audio,
|
||||
canvas,
|
||||
iframe,
|
||||
img,
|
||||
svg,
|
||||
video {
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
/*
|
||||
* Remove default fieldset styles.
|
||||
*/
|
||||
|
||||
fieldset {
|
||||
border: 0;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/*
|
||||
* Allow only vertical resizing of textareas.
|
||||
*/
|
||||
|
||||
textarea {
|
||||
resize: vertical;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Browser Upgrade Prompt
|
||||
========================================================================== */
|
||||
|
||||
.browserupgrade {
|
||||
margin: 0.2em 0;
|
||||
background: #ccc;
|
||||
color: #000;
|
||||
padding: 0.2em 0;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Author's custom styles
|
||||
========================================================================== */
|
||||
|
||||
/* ==========================================================================
|
||||
Helper classes
|
||||
========================================================================== */
|
||||
|
||||
/*
|
||||
* Hide visually and from screen readers
|
||||
*/
|
||||
|
||||
.hidden {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
/*
|
||||
* Hide only visually, but have it available for screen readers:
|
||||
* https://snook.ca/archives/html_and_css/hiding-content-for-accessibility
|
||||
*
|
||||
* 1. For long content, line feeds are not interpreted as spaces and small width
|
||||
* causes content to wrap 1 word per line:
|
||||
* https://medium.com/@jessebeach/beware-smushed-off-screen-accessible-text-5952a4c2cbfe
|
||||
*/
|
||||
|
||||
.sr-only {
|
||||
border: 0;
|
||||
clip: rect(0, 0, 0, 0);
|
||||
height: 1px;
|
||||
margin: -1px;
|
||||
overflow: hidden;
|
||||
padding: 0;
|
||||
position: absolute;
|
||||
white-space: nowrap;
|
||||
width: 1px;
|
||||
/* 1 */
|
||||
}
|
||||
|
||||
/*
|
||||
* Extends the .sr-only class to allow the element
|
||||
* to be focusable when navigated to via the keyboard:
|
||||
* https://www.drupal.org/node/897638
|
||||
*/
|
||||
|
||||
.sr-only.focusable:active,
|
||||
.sr-only.focusable:focus {
|
||||
clip: auto;
|
||||
height: auto;
|
||||
margin: 0;
|
||||
overflow: visible;
|
||||
position: static;
|
||||
white-space: inherit;
|
||||
width: auto;
|
||||
}
|
||||
|
||||
/*
|
||||
* Hide visually and from screen readers, but maintain layout
|
||||
*/
|
||||
|
||||
.invisible {
|
||||
visibility: hidden;
|
||||
}
|
||||
|
||||
/*
|
||||
* Clearfix: contain floats
|
||||
*
|
||||
* For modern browsers
|
||||
* 1. The space content is one way to avoid an Opera bug when the
|
||||
* `contenteditable` attribute is included anywhere else in the document.
|
||||
* Otherwise it causes space to appear at the top and bottom of elements
|
||||
* that receive the `clearfix` class.
|
||||
* 2. The use of `table` rather than `block` is only necessary if using
|
||||
* `:before` to contain the top-margins of child elements.
|
||||
*/
|
||||
|
||||
.clearfix:before,
|
||||
.clearfix:after {
|
||||
content: " ";
|
||||
/* 1 */
|
||||
display: table;
|
||||
/* 2 */
|
||||
}
|
||||
|
||||
.clearfix:after {
|
||||
clear: both;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
EXAMPLE Media Queries for Responsive Design.
|
||||
These examples override the primary ('mobile first') styles.
|
||||
Modify as content requires.
|
||||
========================================================================== */
|
||||
|
||||
@media only screen and (min-width: 35em) {
|
||||
/* Style adjustments for viewports that meet the condition */
|
||||
}
|
||||
|
||||
@media print,
|
||||
(-webkit-min-device-pixel-ratio: 1.25),
|
||||
(min-resolution: 1.25dppx),
|
||||
(min-resolution: 120dpi) {
|
||||
/* Style adjustments for high resolution devices */
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Print styles.
|
||||
Inlined to avoid the additional HTTP request:
|
||||
https://www.phpied.com/delay-loading-your-print-css/
|
||||
========================================================================== */
|
||||
|
||||
@media print {
|
||||
*,
|
||||
*:before,
|
||||
*:after {
|
||||
background: transparent !important;
|
||||
color: #000 !important;
|
||||
/* Black prints faster */
|
||||
box-shadow: none !important;
|
||||
text-shadow: none !important;
|
||||
}
|
||||
a,
|
||||
a:visited {
|
||||
text-decoration: underline;
|
||||
}
|
||||
a[href]:after {
|
||||
content: " (" attr(href) ")";
|
||||
}
|
||||
abbr[title]:after {
|
||||
content: " (" attr(title) ")";
|
||||
}
|
||||
/*
|
||||
* Don't show links that are fragment identifiers,
|
||||
* or use the `javascript:` pseudo protocol
|
||||
*/
|
||||
a[href^="#"]:after,
|
||||
a[href^="javascript:"]:after {
|
||||
content: "";
|
||||
}
|
||||
pre {
|
||||
white-space: pre-wrap !important;
|
||||
}
|
||||
pre,
|
||||
blockquote {
|
||||
border: 1px solid #999;
|
||||
page-break-inside: avoid;
|
||||
}
|
||||
/*
|
||||
* Printing Tables:
|
||||
* https://web.archive.org/web/20180815150934/http://css-discuss.incutio.com/wiki/Printing_Tables
|
||||
*/
|
||||
thead {
|
||||
display: table-header-group;
|
||||
}
|
||||
tr,
|
||||
img {
|
||||
page-break-inside: avoid;
|
||||
}
|
||||
p,
|
||||
h2,
|
||||
h3 {
|
||||
orphans: 3;
|
||||
widows: 3;
|
||||
}
|
||||
h2,
|
||||
h3 {
|
||||
page-break-after: avoid;
|
||||
}
|
||||
}
|
||||
|
349
static/css/normalize.css
vendored
Normal file
349
static/css/normalize.css
vendored
Normal file
@ -0,0 +1,349 @@
|
||||
/*! normalize.css v8.0.1 | MIT License | github.com/necolas/normalize.css */
|
||||
|
||||
/* Document
|
||||
================= |