U 3d1 @sdZddlZddlmZddlZddlZddlmZmZddl Z ddl Z ddl m Z ddl mZddlmZdd l mZdd l mZd d lmZd d lmZd dlmZeddddZeddddZeeZdddddddddddZdddZddZdS) zKDDCUP 99 dataset. A classic dataset for anomaly detection. The dataset page is available from UCI Machine Learning Repository https://archive.ics.uci.edu/ml/machine-learning-databases/kddcup99-mld/kddcup.data.gz N)GzipFile)existsjoin) _fetch_remote)_convert_data_dataframe) get_data_home)RemoteFileMetadata) load_descr)Bunch)check_random_state)shuffleZ kddcup99_dataz.https://ndownloader.figshare.com/files/5976045Z@3b6c942aa0356c0ca35b7b595a26c89d343652c9db428893e7494f837b274292)filenameurlZchecksumZkddcup99_10_dataz.https://ndownloader.figshare.com/files/5976042Z@8045aca0d84e70e622d1148d7df782496f6333bf6eb979a1b0837c42a9fd9561FT)subset data_homer random_state percent10download_if_missing return_X_yas_framecCst|d}t|||d}|j} |j} |j} |j} |dkr| dk} t| }| | ddf}| | }| |ddf}| |}|jd}t |}| d|d}||}||}tj ||f} tj ||f} |dks|d ks|d kr"| ddd fd k} tj | | dd f| | d dff} | dd | d d} | | } t | dddfdjtdd| dddf<t | dddfdjtdd| dddf<t | dddfdjtdd| dddf<|d krB| dddfdk} | | } | | } tj | dddf| dddf| dddff} | d| d| dg} |d kr| dddfdk} | | } | | } tj | dddf| dddf| dddff} | d| d| dg} |dkr"tj | dddf| dddf| dddf| dddff} | d| d| d| dg} |r:t| | |d\} } td}d}|rbtd| | | | \}} } |rp| | fSt| | || | |dS)a Load the kddcup99 dataset (classification). Download it if necessary. ================= ==================================== Classes 23 Samples total 4898431 Dimensionality 41 Features discrete (int) or continuous (float) ================= ==================================== Read more in the :ref:`User Guide `. .. versionadded:: 0.18 Parameters ---------- subset : {'SA', 'SF', 'http', 'smtp'}, default=None To return the corresponding classical subsets of kddcup 99. If None, return the entire kddcup 99 dataset. data_home : str, default=None Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. .. versionadded:: 0.19 shuffle : bool, default=False Whether to shuffle dataset. random_state : int, RandomState instance or None, default=None Determines random number generation for dataset shuffling and for selection of abnormal samples if `subset='SA'`. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. percent10 : bool, default=True Whether to load only 10 percent of the data. download_if_missing : bool, default=True If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site. return_X_y : bool, default=False If True, returns ``(data, target)`` instead of a Bunch object. See below for more information about the `data` and `target` object. .. versionadded:: 0.20 as_frame : bool, default=False If `True`, returns a pandas Dataframe for the ``data`` and ``target`` objects in the `Bunch` returned object; `Bunch` return object will also have a ``frame`` member. .. versionadded:: 0.24 Returns ------- data : :class:`~sklearn.utils.Bunch` Dictionary-like object, with the following attributes. data : {ndarray, dataframe} of shape (494021, 41) The data matrix to learn. If `as_frame=True`, `data` will be a pandas DataFrame. target : {ndarray, series} of shape (494021,) The regression target for each sample. If `as_frame=True`, `target` will be a pandas Series. frame : dataframe of shape (494021, 42) Only present when `as_frame=True`. Contains `data` and `target`. DESCR : str The full description of the dataset. feature_names : list The names of the dataset columns target_names: list The names of the target columns (data, target) : tuple if ``return_X_y`` is True A tuple of two ndarray. The first containing a 2D array of shape (n_samples, n_features) with each row representing one sample and each column representing the features. The second ndarray of shape (n_samples,) containing the target samples. .. versionadded:: 0.20 r)rrrZSAsnormal.Nri1 ZSFhttpsmtp r g?F)copyr shttpssmtp)rz kddcup99.rstfetch_kddcup99)datatargetframe target_names feature_namesZDESCR)r_fetch_brute_kddcup99r!r"r%r$npZ logical_notshaper randintZr_Zc_logastypefloatshuffle_methodr rr )rrrrrrrrkddcup99r!r"r%r$stZnormal_samplesZnormal_targetsZabnormal_samplesZabnormal_targetsZn_samples_abnormalrZfdescrr#r2>/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/datasets/_kddcup99.pyr 1s_   &000 4 4 B r c*Cst|d}d}|r&t|d|}t}nt|d|}t}t|d}t|d}t|}dtfdd d d tfd tfd tfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfdtfd tfd!tfd"tfd#tfd$tfd%tfd&tfd'tfd(tfd)tfd*tfd+tfd,tfd-tfd.tfd/tfd0g*} d1d2| D} | d3} | d4d3} |rzt|} t|}Wn:t k r}zt d5t |d6|W5d4}~XYnXn8|rt |t d7|jt||d8t| }t d9t||j}t|d:d;}g}|D]&}|}||dq&|t d?t|tj|td@}t dAD],}|d4d4|f!|||d4d4|f<q|d4d4d4d3f} |d4d4d3f}tj"| |dBdCtj"||dBdCnt dDt#| || | gdES)FaLoad the kddcup99 dataset, downloading it if necessary. Parameters ---------- data_home : str, default=None Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. download_if_missing : bool, default=True If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site. percent10 : bool, default=True Whether to load only 10 percent of the data. Returns ------- dataset : :class:`~sklearn.utils.Bunch` Dictionary-like object, with the following attributes. data : ndarray of shape (494021, 41) Each row corresponds to the 41 features in the dataset. target : ndarray of shape (494021,) Each value corresponds to one of the 21 attack types or to the label 'normal.'. feature_names : list The names of the dataset columns target_names: list The names of the target columns DESCR : str Description of the kddcup99 dataset. rz-py3Z kddcup99_10r.Zsamplestargetsduration)Z protocol_typeZS4)ZserviceZS11)flagZS6Z src_bytesZ dst_bytesZlandZwrong_fragmentZurgentZhotZnum_failed_loginsZ logged_inZnum_compromisedZ root_shellZ su_attemptedZnum_rootZnum_file_creationsZ num_shellsZnum_access_filesZnum_outbound_cmdsZ is_host_loginZis_guest_logincountZ srv_countZ serror_rateZsrv_serror_rateZ rerror_rateZsrv_rerror_rateZ same_srv_rateZ diff_srv_rateZsrv_diff_host_rateZdst_host_countZdst_host_srv_countZdst_host_same_srv_rateZdst_host_diff_srv_rateZdst_host_same_src_port_rateZdst_host_srv_diff_host_rateZdst_host_serror_rateZdst_host_srv_serror_rateZdst_host_rerror_rateZdst_host_srv_rerror_rate)labelsZS16cSsg|] }|dqS)rr2).0cr2r2r3 Fsz)_fetch_brute_kddcup99..Nz7The cache for fetch_kddcup99 is invalid, please delete z! and run the fetch_kddcup99 againzDownloading %s)dirnamezextracting archiver1)rmode ,zextraction done)dtype*r)compressz1Data not found and `download_if_missing` is False)r!r"r%r$)$rrARCHIVE_10_PERCENTARCHIVErintr,joblibload ExceptionIOErrorstr_mkdirploggerinforrr'rBdebugrr readlinesdecodeappendreplacesplitcloseosremoveZasarrayobjectranger+dumpr )rrrZ dir_suffixZ kddcup_dirarchiveZ samples_pathZ targets_path availabledtZ column_namesr$r%XyeZDT archive_pathfile_ZXylinejr2r2r3r&s#   -           *r&c CsDzt|Wn0tk r>}z|jtjkr.W5d}~XYnXdS)zgEnsure directory d exists (like mkdir -p on Unix) No guarantee that the directory is writable. N)rWmakedirsOSErrorerrnoEEXIST)drar2r2r3rM{s  rM)NTT) __doc__rhgziprloggingrWos.pathrrZnumpyr'rH_baserrr@rr r utilsr r rr-rFrE getLogger__name__rNr r&rMr2r2r2r3sN            8