Remeis-Wiki - User contributions [en]

TODO

2018-05-08T14:38:14Z

Bick: Redirected page to Internal TODO / "to discuss" area

#REDIRECT[[Internal TODO / "to discuss" area]]

Q&A

2018-05-08T14:36:33Z

Bick: Redirected page to Questions & Answers

#REDIRECT [[Questions & Answers]]

Popular science: Black Holes

2018-05-08T14:30:16Z

Bick: Created page with "'''Questions we might get asked about black holes''' '''1) Why use X-rays to study black holes?''' X-ray emission is usually associated with extremely high temperatures in p..."

'''Questions we might get asked about black holes'''

'''1) Why use X-rays to study black holes?'''

X-ray emission is usually associated with extremely high
temperatures in plasmas/gas.

In black hole systems, X-rays are generated extremely close to the
black hole, and are a prime observational signature of superheated gas
within tens of gravitational radii from the black hole. Studying X-ray
emission means we're guaranteed emission originating from within the
region of extreme gravity.

Furthermore, unlike optical, UV, or EUV photons, X-ray photons
typically do not suffer from strong interstellar absorption during
their journey out of the AGN, through the host galaxy and intergalactic
space, and through the Milky Way.

'''2) Why are black holes important to study?'''

They are laboratories for extreme gravity, and conditions that we
cannot replicate on Earth. They are excellent test-beds for
General Relativity theory.

In the case of supermassive black holes, they're linked to the
formation and evolution of the galaxies in which they reside. That
is, the presence and activity (electromagnetic radiation and
outflows) of a SMBH can influence its host on large length scales.

(See also: http://chandra.as.utexas.edu/~kormendy/stardate.html,
sect. VIII)

In addition, the process of "accretion" onto all the black holes in the
Universe is a major physical process, responsible for a tremendous
amount of EM radiation and a lot of mass transferred from galaxies
to SMBHs.

'''(someone should add something about specific connections between stellar-mass black holes and evolution of massive stars) '''

'''3) How big are black holes?'''

Black holes themselves are singularities -- infinitesimally small
point sources where all the mass is contained.

However, we can talk about the event horizon or "point of no return"
("Schwarzschild radius", if the black hole is non-rotating) as a characteristic length scale.
Within this radius, nothing can escape, no matter its velocity;
even light cannot escape. Outside the event horizon, things still
have the ability to escape if their velocity is high enough.

The event horizon radius scales with black hole mass:

1 RSch = 3 km × Mass(Msun)

For stellar-mass black holes:

3 Msun: 1 RSch = 9 km
(e.g., a diameter of 18 km means the event horizon would fit between Bamberg and Eggolsheim)

10 Msun: 1 RSch = 30 km (e.g., a diameter of 60 km means the event horizon
would fit between Bamberg and central Nuernberg)

For supermassive black holes, the event horizon radius is:

106 Msun: 1 RSch = 3×106 km = 0.02 AU = 4.3× the Sun's radius

107 Msun: 1 RSch = 3×107 km = 0.2 AU = ~half the average Sun-Mercury distance

108 Msun: 1 RSch = 3×108 km = 2.0 AU = 1.3× the Sun-Mars distance

109 Msun: 1 RSch = 3×109 km = 20.0 AU = roughly the Sun-Uranus distance

'''4) What's the evidence that black holes exist?'''

There is typically a strong gravitational effect on nearby gas or
stars, causing those gas/stars to move at speeds much higher than if
there were no massive, dense object nearby.

For a stellar-mass black hole in a binary system, we observe Doppler
shifting of the optical companion star as they orbit each other.

(See also http://imagine.gsfc.nasa.gov/docs/science/know_l2/black_holes.html)

For supermassive black holes, we can study the kinematics of stars.
For the Milky Way, we can track individual stars as they orbit around Sgr A*

(See http://www.astro.ucla.edu/~ghezgroup/gc/pictures/orbitsMovie.shtml ).

We also analyze the velocity dispersions of stars in the bulges of
other galaxies (they correlate well with black hole mass, a.k.a. the
"MBH--sigma* relation" ; see

http://blackholes.stardate.org/resources/article.php?p=birth-of-black-holes for more info.)

We can also identify circumnuclear gas disks around supermassive black
holes via their emission line profiles. Atomic and molcular hydrogren
in the optical and IR bands, CO emission in the mm band, and
mega-masers in the radio band indicate gas disks orbiting within
100-300 pc from black holes in (very roughly) a couple dozen nearby
galaxies.

(e.g., http://ned.ipac.caltech.edu/level5/March01/Ho/Ho4.html,
http://arxiv.org/abs/1301.7184, http://astro.ucsc.edu/~nng/webtalks/hicks_poster.pdf,
http://www.nature.com/nature/journal/v494/n7437/full/nature11819.html)

For both types of black holes: we observe copious amounts of lots of
X-rays emitted as gravitational potential energy is converted into EM
radiation, as well as rapid flux variability. We also observe Fe
K-alpha emission line broadening that indicates gas moving at extreme
velocities.

(See also http://imagine.gsfc.nasa.gov/docs/ask_astro/answers/970805b.html)

'''5) How many black holes are there?'''

Let's break this question up into three parts:

# How many stellar-mass black holes in the Milky Way are we aware of?
#* So far, at least 20 actively accreting X-ray binary systems have been confirmed to contain an object that is compact and massive enough to be a black hole. However, an additional ~20 systems are suspected of hosting black holes (tentative or moderate-quality evidence). See, e.g., http://mintaka.sdsu.edu/faculty/orosz/web/
#* There could be hundreds of additional black hole binary systems in the Milky Way that we haven't detected/identified yet (e.g., if they are not accreting, and thus producing X-rays, at this time).
# How many supermassive black holes are there?
#* The answer could be the same as that for the question 'How many large galaxies are there?'
#* The Sloan Digital Sky Survey has identified one million quasars so far....
# How many stellar-mass black holes do we know of in other galaxies?
#* We have identified so far (a few dozen? So far I'm not finding precise info.) Ultra-Luminous X-ray sources (ULXs) in other nearby large galaxies. They may be powered by accretion onto steller-mass black holes, but may also host "intermediate-mass black holes" (102-104 Msun).
#* IMBHs may also exist in globular clusters.

[[Category:Internal]]

ERosita Questions & Answers

2018-05-08T14:06:32Z

Bick:

====== Questions & Answers ======
*Q: Do we need an additional tool to preprocess the latest data set in order to get a quick look at the current status of the instruments or is it sufficient to integrate that in the preprocessing pipeline with a prioritizing task?
*:-A: A different tool would be best suited, because the preprocessing would get complicated if we try to integrate that in the preprocessing pipeline. This tool would indeed be some kind of "RTA - Real Time Analysis".

*Q: Will we have a different preprocessing pipeline for the different kinds of data we get (Housekeeping data, Event data, Maps)?
*:-A: Housekeeping data as well as Event data will use the same preprocessing pipeline, only the check for a complete data sample will probably be different. As for the maps we still have to decide if they need any kind of preprocessing or if we just forward them to the next steps.

*Q: Is a completeness check based on continuous time stamps possible or will the data be split with the Russians before we process the data?
*:-A: The data will most probably not be split before, otherwise a lot of things would not make sense. But this needs clarification.

*Q: Do we have to remove header keywords, which we added to the fits files after preprocessing or other tasks?
*:-A: In most cases NO! Only if this leads to misinterpreting or understanding of keywords.

*Q: Do we need a tool to create GTI based upon the status of the instrument e.g. temperature is too high - especially for the Bad Pixel Detection?
*:-A: No that is not necessary. On the one hand we should have a person monitoring the values and thus deciding on the Bad Pixels and on the other hand this should be a part of the SASS (@Ingo could you pls correct me here?).

*Q: Do we need a tool which looks for annealing processes?
*:-A: No thats not necessary at the moment.

*Q: Shall we prevent the preprocessing pipe to run in multiple instances?
*:-A: Yes, this could lead to a lot of problems with the pipe itself as well as with the ftools used by the pipe.

*Q: How do we prevent pipes to run more than once simultaneously?
*:-A: Not decided yet. One possible way would be to do it like Firefox and create a kind of log file which can be checked and then either be allowed to run or not.

*Q: How shall the NRTA be started?
*:-A: The NRTA pipe shall be imbedded in a perl/shell/c script/program, that continuously reads the input folder and runs the perl script fpipegui.pl with different parameters based upon the new files.

*Q: How do we deal with data which arrives after a proposed waiting time?
*:-A: This still needs some discussion.

*Q: What criteria do we use to make sure that data was correctly copied to a different directory?
*:-A: This still is up to discussion. One way would be to calculate the checksum and make a verification test of the checksum. But this does not solve the problem if the checksum verification is correct but the wrong data was copied or an old file was still in the directory and was not deleted/replaced. One way would be to also check if the two checksums are the same.

The whole page is subject to change in time.

[[Category:EROSITA]]

ERosita Preprocessing

2018-05-08T14:06:13Z

Bick:

====== Preprocessing ======
This is still under construction. Below find the current files regarding the preprocessing pipeline.

It starts with the main.pdf. The other files are marked in the main program. One correction the diagram for NextEroday from the main.pdf is called initialise_prepro here. This is subject to change in the future.

In addition there are no boxes for warnings and logs included as well as a check if a rm of a file was successful.
Reason for the concept (short):
The basic idea behind this approach was, to make pipeline work again event if was interrupted without processing lots of the files again. We also avoided the useage of logfiles to continue with our processing, so that the pipe can also be run on a different compute, with just the input folder and the archive.

{{:intern:erosita:main.pdf}} \\
{{:intern:erosita:check_nrta_cp.pdf}} \\
{{:intern:erosita:completeness_check.pdf}} \\
{{:intern:erosita:corrupt_check.pdf}} \\
{{:intern:erosita:create_files.pdf}} \\
{{:intern:erosita:get_data.pdf}} \\
{{:intern:erosita:initialise_prepro.pdf}} \\
{{:intern:erosita:merge_data.pdf}} \\

TODO / TO THINK ABOUT:
* GTI Extension is in every file thats why ftmerge might not be the right tool
* GTI Extensions have to be verified
* Frames can be sent twice, so before merge calculate exposure of the GTI Extensions and the exposure in data. If they are different make a temporary exclude GTI file.
* Real Problems arise if frames are split into two or more files. Have to be checked too.
* system( command ) is not the right way to execute programs use the gnu exec commands instead http://www.gnu.org/s/libc/manual/html_node/Executing-a-File.html

[[Category:EROSITA]]

Internal TODO / "to discuss" area

2018-05-08T14:05:42Z

Bick:

===== internal TODO / "to discuss" area =====

*libraries: FITS images -> support for more than **two** dimensions required?
*error handling: should we use the built-in GNU C signaling formalism (see [[http://www.cs.utah.edu/dept/old/texinfo/glibc-manual-0.02/library_21.html|Signal handling]])? That would provide the advantage that we could then easily define different error handling functions.
*will pre-NRTA-files contain a CCD_NR column or will there be 7 different files, one for each camera?
*__there will be one file for each camera. A merge of all files will probably be done by the SASS which will then require a CCD_NR column__

[[Category:EROSITA]]

EROSITA

2018-05-08T14:05:25Z

Bick:

====== eROSITA ======
This site shall contain useful information and links for all the people who are interested in eROSITA.

In Addition a side of information regarding eROSITA development is maintained by Hermann Brunner: [[http://www.mpe.mpg.de/erosita/internal/SASS-devel/| Link]]

===== Information =====

===== Programming Topics in Bamberg =====
*Converting the telemetry data stream into the FITS file format
*Simulation
*Preprocessing
*NRTA:Health
*NRTA:Science

===== Misc =====

[[Category:EROSITA]]

Internal TODO / "to discuss" area

2018-05-08T13:58:07Z

Bick: Created page with " Category:EROSITA ===== internal TODO / "to discuss" area ===== *libraries: FITS images -> support for more than **two** dimensions required? *error handling: should we..."

[[Category:EROSITA]]
===== internal TODO / "to discuss" area =====

*libraries: FITS images -> support for more than **two** dimensions required?
*error handling: should we use the built-in GNU C signaling formalism (see [[http://www.cs.utah.edu/dept/old/texinfo/glibc-manual-0.02/library_21.html|Signal handling]])? That would provide the advantage that we could then easily define different error handling functions.
*will pre-NRTA-files contain a CCD_NR column or will there be 7 different files, one for each camera?
*__there will be one file for each camera. A merge of all files will probably be done by the SASS which will then require a CCD_NR column__

EROSITA

2018-05-08T13:55:12Z

Bick: Created page with " Category:EROSITA ====== eROSITA ====== This site shall contain useful information and links for all the people who are interested in eROSITA. In Addition a side of info..."

[[Category:EROSITA]]
====== eROSITA ======
This site shall contain useful information and links for all the people who are interested in eROSITA.

In Addition a side of information regarding eROSITA development is maintained by Hermann Brunner: [[http://www.mpe.mpg.de/erosita/internal/SASS-devel/| Link]]

===== Information =====

===== Programming Topics in Bamberg =====
*Converting the telemetry data stream into the FITS file format
*Simulation
*Preprocessing
*NRTA:Health
*NRTA:Science

===== Misc =====

ERosita Questions & Answers

2018-05-08T13:52:12Z

Bick:

[[Category:EROSITA]]
====== Questions & Answers ======
*Q: Do we need an additional tool to preprocess the latest data set in order to get a quick look at the current status of the instruments or is it sufficient to integrate that in the preprocessing pipeline with a prioritizing task?
*:-A: A different tool would be best suited, because the preprocessing would get complicated if we try to integrate that in the preprocessing pipeline. This tool would indeed be some kind of "RTA - Real Time Analysis".

*Q: Will we have a different preprocessing pipeline for the different kinds of data we get (Housekeeping data, Event data, Maps)?
*:-A: Housekeeping data as well as Event data will use the same preprocessing pipeline, only the check for a complete data sample will probably be different. As for the maps we still have to decide if they need any kind of preprocessing or if we just forward them to the next steps.

*Q: Is a completeness check based on continuous time stamps possible or will the data be split with the Russians before we process the data?
*:-A: The data will most probably not be split before, otherwise a lot of things would not make sense. But this needs clarification.

*Q: Do we have to remove header keywords, which we added to the fits files after preprocessing or other tasks?
*:-A: In most cases NO! Only if this leads to misinterpreting or understanding of keywords.

*Q: Do we need a tool to create GTI based upon the status of the instrument e.g. temperature is too high - especially for the Bad Pixel Detection?
*:-A: No that is not necessary. On the one hand we should have a person monitoring the values and thus deciding on the Bad Pixels and on the other hand this should be a part of the SASS (@Ingo could you pls correct me here?).

*Q: Do we need a tool which looks for annealing processes?
*:-A: No thats not necessary at the moment.

*Q: Shall we prevent the preprocessing pipe to run in multiple instances?
*:-A: Yes, this could lead to a lot of problems with the pipe itself as well as with the ftools used by the pipe.

*Q: How do we prevent pipes to run more than once simultaneously?
*:-A: Not decided yet. One possible way would be to do it like Firefox and create a kind of log file which can be checked and then either be allowed to run or not.

*Q: How shall the NRTA be started?
*:-A: The NRTA pipe shall be imbedded in a perl/shell/c script/program, that continuously reads the input folder and runs the perl script fpipegui.pl with different parameters based upon the new files.

*Q: How do we deal with data which arrives after a proposed waiting time?
*:-A: This still needs some discussion.

*Q: What criteria do we use to make sure that data was correctly copied to a different directory?
*:-A: This still is up to discussion. One way would be to calculate the checksum and make a verification test of the checksum. But this does not solve the problem if the checksum verification is correct but the wrong data was copied or an old file was still in the directory and was not deleted/replaced. One way would be to also check if the two checksums are the same.

The whole page is subject to change in time.

ERosita Questions & Answers

2018-05-08T13:51:50Z

Bick:

[[Category:EROSITA]]

====== Questions & Answers ======
*Q: Do we need an additional tool to preprocess the latest data set in order to get a quick look at the current status of the instruments or is it sufficient to integrate that in the preprocessing pipeline with a prioritizing task?
*:-A: A different tool would be best suited, because the preprocessing would get complicated if we try to integrate that in the preprocessing pipeline. This tool would indeed be some kind of "RTA - Real Time Analysis".

*Q: Will we have a different preprocessing pipeline for the different kinds of data we get (Housekeeping data, Event data, Maps)?
*:-A: Housekeeping data as well as Event data will use the same preprocessing pipeline, only the check for a complete data sample will probably be different. As for the maps we still have to decide if they need any kind of preprocessing or if we just forward them to the next steps.

*Q: Is a completeness check based on continuous time stamps possible or will the data be split with the Russians before we process the data?
*:-A: The data will most probably not be split before, otherwise a lot of things would not make sense. But this needs clarification.

*Q: Do we have to remove header keywords, which we added to the fits files after preprocessing or other tasks?
*:-A: In most cases NO! Only if this leads to misinterpreting or understanding of keywords.

*Q: Do we need a tool to create GTI based upon the status of the instrument e.g. temperature is too high - especially for the Bad Pixel Detection?
*:-A: No that is not necessary. On the one hand we should have a person monitoring the values and thus deciding on the Bad Pixels and on the other hand this should be a part of the SASS (@Ingo could you pls correct me here?).

*Q: Do we need a tool which looks for annealing processes?
*:-A: No thats not necessary at the moment.

*Q: Shall we prevent the preprocessing pipe to run in multiple instances?
*:-A: Yes, this could lead to a lot of problems with the pipe itself as well as with the ftools used by the pipe.

*Q: How do we prevent pipes to run more than once simultaneously?
*:-A: Not decided yet. One possible way would be to do it like Firefox and create a kind of log file which can be checked and then either be allowed to run or not.

*Q: How shall the NRTA be started?
*:-A: The NRTA pipe shall be imbedded in a perl/shell/c script/program, that continuously reads the input folder and runs the perl script fpipegui.pl with different parameters based upon the new files.

*Q: How do we deal with data which arrives after a proposed waiting time?
*:-A: This still needs some discussion.

*Q: What criteria do we use to make sure that data was correctly copied to a different directory?
*:-A: This still is up to discussion. One way would be to calculate the checksum and make a verification test of the checksum. But this does not solve the problem if the checksum verification is correct but the wrong data was copied or an old file was still in the directory and was not deleted/replaced. One way would be to also check if the two checksums are the same.

The whole page is subject to change in time.

ERosita Questions & Answers

2018-05-08T13:44:30Z

Bick: Created page with " Category:EROSITA ====== Questions & Answers ====== Q: Do we need an additional tool to preprocess the latest data set in order to get a quick look at the current stat..."

[[Category:EROSITA]]

====== Questions & Answers ======
Q: Do we need an additional tool to preprocess the latest data set in order to get a quick look at the current status of the instruments or is it sufficient to integrate that in the preprocessing pipeline with a prioritizing task?

A: A different tool would be best suited, because the preprocessing would get complicated if we try to integrate that in the preprocessing pipeline. This tool would indeed be some kind of "RTA - Real Time Analysis".

- Q: Will we have a different preprocessing pipeline for the different kinds of data we get (Housekeeping data, Event data, Maps)?

A: Housekeeping data as well as Event data will use the same preprocessing pipeline, only the check for a complete data sample will probably be different. As for the maps we still have to decide if they need any kind of preprocessing or if we just forward them to the next steps.

- Q: Is a completeness check based on continuous time stamps possible or will the data be split with the Russians before we process the data?

A: The data will most probably not be split before, otherwise a lot of things would not make sense. But this needs clarification.

- Q: Do we have to remove header keywords, which we added to the fits files after preprocessing or other tasks?

A: In most cases NO! Only if this leads to misinterpreting or understanding of keywords.

- Q: Do we need a tool to create GTI based upon the status of the instrument e.g. temperature is too high - especially for the Bad Pixel Detection?

A: No that is not necessary. On the one hand we should have a person monitoring the values and thus deciding on the Bad Pixels and on the other hand this should be a part of the SASS (@Ingo could you pls correct me here?).

- Q: Do we need a tool which looks for annealing processes?

A: No thats not necessary at the moment.
- Q: Shall we prevent the preprocessing pipe to run in multiple instances?

A: Yes, this could lead to a lot of problems with the pipe itself as well as with the ftools used by the pipe.

- Q: How do we prevent pipes to run more than once simultaneously?

A: Not decided yet. One possible way would be to do it like Firefox and create a kind of log file which can be checked and then either be allowed to run or not.

- Q: How shall the NRTA be started?

A: The NRTA pipe shall be imbedded in a perl/shell/c script/program, that continuously reads the input folder and runs the perl script fpipegui.pl with different parameters based upon the new files.

- Q: How do we deal with data which arrives after a proposed waiting time?

A: This still needs some discussion.

- Q: What criteria do we use to make sure that data was correctly copied to a different directory?

A: This still is up to discussion. One way would be to calculate the checksum and make a verification test of the checksum. But this does not solve the problem if the checksum verification is correct but the wrong data was copied or an old file was still in the directory and was not deleted/replaced. One way would be to also check if the two checksums are the same.

The whole page is subject to change in time.

ERosita Preprocessing

2018-05-08T13:36:39Z

Bick: Created page with " Category:EROSITA ====== Preprocessing ====== This is still under construction. Below find the current files regarding the preprocessing pipeline. It starts with the m..."

[[Category:EROSITA]]

====== Preprocessing ======
This is still under construction. Below find the current files regarding the preprocessing pipeline.

It starts with the main.pdf. The other files are marked in the main program. One correction the diagram for NextEroday from the main.pdf is called initialise_prepro here. This is subject to change in the future.

In addition there are no boxes for warnings and logs included as well as a check if a rm of a file was successful.
Reason for the concept (short):
The basic idea behind this approach was, to make pipeline work again event if was interrupted without processing lots of the files again. We also avoided the useage of logfiles to continue with our processing, so that the pipe can also be run on a different compute, with just the input folder and the archive.

{{:intern:erosita:main.pdf}} \\
{{:intern:erosita:check_nrta_cp.pdf}} \\
{{:intern:erosita:completeness_check.pdf}} \\
{{:intern:erosita:corrupt_check.pdf}} \\
{{:intern:erosita:create_files.pdf}} \\
{{:intern:erosita:get_data.pdf}} \\
{{:intern:erosita:initialise_prepro.pdf}} \\
{{:intern:erosita:merge_data.pdf}} \\

TODO / TO THINK ABOUT:
* GTI Extension is in every file thats why ftmerge might not be the right tool
* GTI Extensions have to be verified
* Frames can be sent twice, so before merge calculate exposure of the GTI Extensions and the exposure in data. If they are different make a temporary exclude GTI file.
* Real Problems arise if frames are split into two or more files. Have to be checked too.\\
* system( command ) is not the right way to execute programs use the gnu exec commands instead http://www.gnu.org/s/libc/manual/html_node/Executing-a-File.html