Special Session on Mining the sky: knowledge discovery in big and complex astronomical data sets and data streams

Astronomical observations produce some of the largest “big data” today through a new generation of telescopes, and by next-generation telescopes planned to be launched in less than a decade. While these data sets exhibit the usual challenges associated with big data (immense data volumes, high dimensionality, high complexity, disparate variables, etc.) there are new problems such as data types that represent a whole new nature and level of complexity. Information extraction from astronomical observations also warrants specific focus. For example, classification, data mining and pattern discovery must produce very precise estimates (more precise than, e.g., in terrestrial remote sensing) from imagery of extremely low signal-to-noise ratio. Another example is the need to deliver some results from multi-terabyte-size data in (near-)real-time to guide the next day’s observation of interesting objects and best exploit short windows of observing times. A distinguishing factor of astronomical data sets is that, unlike, e.g., in medical or social domains, there are strict laws of physics behind the data production which can often be assimilated into machine learning to improve over general off-the-shelf state-of-the-art. A surge of discussion of specific problems and needs for collaborations between astronomers and computational / data experts has started in recent years. This Special Session will further this process through an IEEE platform. It aims to engage the computational community in solutions to problems modern astronomy faces in turning the sky-full of inexhaustible stream of data into reliable knowledge at an accelerated rate. Machine learning and data mining, computational intelligence approaches in general, are in high demand but as yet not sufficiently exploited. The astronomical community is reaching out to engage expertise in these areas while members of the IEEE community are interested to tackle extreme challenges involved in astronomy projects. Showing the astronomical community successful results from astronomical data obtained by advanced Computational Intelligence (CI) methods is critical because, presently, astronomers in general do not know the methods developed and used in the CI community. But it is equally important to make CI experts (engineers, computer scientists, statisticians …) aware of the existence of compelling scientific problems that require expert applications of state-of-the-art techniques and algorithms developed by the CI community, or motivate developments of new methods. By soliciting papers from author groups representing both communities this special session will effectively serve these objectives, supporting astronomy’s science goals as well as to CI developments in a mutually beneficial way.