API 활용법 (http통신, Json Parsing, 비동기화)

API란?

  Application programming Interface, 어플리케이션의 프로그래밍 통신수단

  일반적으로 OPEN API는 웹서핑과 유사하게 HTTP프로토콜의 request와 response 방식으로 이루어진다. 단지, 전달되는 컨텐츠가 html인지 혹은 데이터(json 등)인지의 차이가 있음. 아래 비교내용 참고.

• Web Browsing:
  • Purpose: The primary purpose of using a URL in web browsing is to locate and retrieve a web page. This page is typically rendered visually in a web browser for user interaction.
  • Content Type: URLs in web browsing usually lead to HTML documents that might also include CSS for styling and JavaScript for interactivity.
•  API Requests:
  • Purpose: In the case of APIs, the URL is used to access various functionalities of an application or service programmatically. The purpose here is not to display content but to perform operations (like retrieving data, submitting data, updating or deleting data) that are part of a larger software function.
  • Content Type: API URLs often respond with data in formats like JSON or XML, which are meant to be processed by software rather than directly read by humans.

사용할 API선택시 고려사항

API documentation 상세확인을 통해서 아래와 같은 질문에 답해본다.

• API가 내가 원하는 얼만큼의 세부적인 query가 가능한가 ? (request)
• API가 사용하기는 편한가? 유료인가 혹은 무료인가
• API의 호출결과가 쓸만한지? (response)
  • 데이터 타입은 사용할만한지? 
  • 제공하는 데이터가 얼마나 상세한가?

API 요청방식 :  (Pure JAVA) HttpURLConnection

출처 : 기상청_단기예보 ((구)_동네예보) 조회서비스 https://www.data.go.kr/data/15084084/openapi.do

import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import java.io.BufferedReader;
import java.io.IOException;

public class Main {
    public static void main(String[] args) throws IOException {
        // Stringbuilder을 사용하여 URL에다 ?를 붙이고 필요한 parameter key와 value를 붙임
        StringBuilder urlBuilder = new StringBuilder("http://apis.data.go.kr/1360000/VilageFcstInfoService_2.0/getUltraSrtNcst"); /*URL*/
        urlBuilder.append("?" + URLEncoder.encode("serviceKey","UTF-8") + "=tQqvr9aPLijnRiuJq0BR2gXuUYRTFDE919UCE4MnQ4iaDl20qpXFeLPQObvamAp2YtsTPy6PSZJAGbPZ%2Fydihw%3D%3D"); /*Service Key*/
        urlBuilder.append("&" + URLEncoder.encode("pageNo","UTF-8") + "=" + URLEncoder.encode("1", "UTF-8")); /*페이지번호*/
        urlBuilder.append("&" + URLEncoder.encode("numOfRows","UTF-8") + "=" + URLEncoder.encode("1000", "UTF-8")); /*한 페이지 결과 수*/
        urlBuilder.append("&" + URLEncoder.encode("dataType","UTF-8") + "=" + URLEncoder.encode("XML", "UTF-8")); /*요청자료형식(XML/JSON) Default: XML*/
        urlBuilder.append("&" + URLEncoder.encode("base_date","UTF-8") + "=" + URLEncoder.encode("20240703", "UTF-8")); /*‘21년 6월 28일 발표*/
        urlBuilder.append("&" + URLEncoder.encode("base_time","UTF-8") + "=" + URLEncoder.encode("0600", "UTF-8")); /*06시 발표(정시단위) */
        urlBuilder.append("&" + URLEncoder.encode("nx","UTF-8") + "=" + URLEncoder.encode("55", "UTF-8")); /*예보지점의 X 좌표값*/
        urlBuilder.append("&" + URLEncoder.encode("ny","UTF-8") + "=" + URLEncoder.encode("127", "UTF-8")); /*예보지점의 Y 좌표값*/
        System.out.println(urlBuilder.toString());

        // 완성된 String을 URL class를 사용하여 url개체를 만듬
        URL url = new URL(urlBuilder.toString());
        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        conn.setRequestMethod("GET");
        conn.setRequestProperty("Content-type", "application/json");
        System.out.println("Response code: " + conn.getResponseCode());
        BufferedReader rd;
        if(conn.getResponseCode() >= 200 && conn.getResponseCode() <= 300) {
            rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
        } else {
            rd = new BufferedReader(new InputStreamReader(conn.getErrorStream()));
        }
        StringBuilder sb = new StringBuilder();
        String line;
        while ((line = rd.readLine()) != null) {
            sb.append(line);
        }
        rd.close();
        conn.disconnect();
        System.out.println(sb.toString());
    }
}

 

  본 예시코드는 일반적인 HTTP통신(서블릿, 스프링 등) 방법과 다르게, Java의 built-in method인 java.net을 사용하였다. Java.net은 다른 http관련 package보다 추상화정도가 낮아 더욱 세세한 http 통신의 조정이 가능하다. http 개념을 복습하는 셈치고 각 라이브러리와 단계를 학습하자.

• 1. URLEncoder.encode :  URL을 만들기 위해서 UTF를 ASCII로 변환하는 작업을 한다. ASCII로 표현가능한 UTF는 그대로 유지하고 아래의 두가지 경우 변환작업을 한다. 변환시에는 % + encoded 16진수번호를 사용해서 
  • A. special characters that have special meanings. Parameter를 표기하는 기호라면 encoding하지않고 값이라면 encoding필요함.
    • The question mark (?) is used to indicate the start of a query string.
    • The ampersand (&) is used to separate query parameters.
    • The equals sign (=) is used to assign values to parameters.
    • empty space can not be included as query parameter
    • # indicates a fragment identifier.
  • B. Character fall outside the ASCII character set such as 한국어.
    
    • 안녕하세요 : %EC%95%88%EB%85%95%ED%95%98%EC%84%B8%EC%9A%94
• 2. URL 클래스를 통해서 url object가 생성되어 http 연결을 준비함.
  • Parsing and Handling: The URL class can parse and decompose a URL string into its components (protocol, host, port, path, query, etc.)
  • Connection Setup: It is used to establish a connection to the resource specified by the URL, typically through methods that open streams for reading from or writing to the target.
• 3. An HttpURLConnection instance is then opened(created) from this URL.
  • HttpURLConnection is to configure http method, key for header property.
  • Lazy Execution: The actual connection to the server (i.e., TCP handshake, SSL handshake if HTTPS, and HTTP request initiation) occurs later when you explicitly call methods that require a live connection, such as: getInputStream(), connect(),  getResponseCode()
• 4. HTTP Response 처리
  • Data Reception: The servlet reads the JSON data sent by the client.
  • JSON Parsing: Parses the JSON to extract the user data.
  • Response Creation: Creates a JSON response with a message indicating successful processing.
  • Sending Response: Sets the content type of the response to application/json and sends the JSON response back to the client.

API 요청방식 :  WebClient

Spring에서 가장 최신이며 보편적으로 사용하는 WebClient를 사용해보자. WebClient를 사용하기 위해서는

'spring-boot-starter-webflux',를 추가해야한다. webflux는 비동기화를 지원하는 dependency이지만 Webclient만을 사용하기위해서 추가하여도 무관하다.

public class HeritageApi {

    public String externalAPI(){

    WebClient client = WebClient.create("https://www.khs.go.kr");
    String response = client.get()
            .uri("/cha/SearchKindOpenapiList.do")
            .retrieve()
            .bodyToMono(String.class)
            .block();  // Synchronous call, use `block()` in non-reactive code

        return response;
    }

}

 

WebClient Method

 

create와 get을 분리해서 하는 것은 동일한 웹사이트에서 여러번 요청을 할 때, 요청마다 instance를 생성하지 않도록 하기 위함. Webclient는 비동기를 위해서 사용하는(mono 사용) 메소드이나 현재는 싱글스레드 상태를 유지하기 위해서 .block()을 사용해두었음.

 

• client.get(): Specifies that this will be a GET request.
• .uri("/cha/SearchKindOpenapiList.do"): Appends the URI path to the base URL.
  • "?pageUnit=20&pageIndex=600&ccbaCncl=N" 이런 uri에 변수를 넣어서 조율하기위해서는 uri Builder를 사용하는 것이 가시성이 좋다. 

.uri(uriBuilder -> uriBuilder
    .path("/cha/SearchKindOpenapiList.do")
    .queryParam("pageUnit", 300)
    .queryParam("pageIndex", pageNum)
    .queryParam("ccbaCncl", "N")
    .build())

 

• .retrieve(): Sends the request and starts retrieving the response.
• .bodyToMono(String.class): Converts the response body to a Mono<String> that represents the XML response.
  • Why use Mono? : A Mono allows non-blocking processing, meaning that WebClient will continue with other work while it waits for the response to arrive. The actual value is only emitted when the response completes.
• .block(): This is a blocking operation that waits for the response to complete and returns the body as a String. Since this is used in synchronous code, it's necessary here, but if you're working in a reactive context, you would avoid using block().

 

API 요청방식 : Performance Consideration 

 

Blocking vs Non-Blocking (Synchronous vs Asynchronous) :

•  Concept Explanation : In a typical blocking I/O server, each incoming client connection is handled by a separate thread. (Creating a new thread for every connection can become expensive, Servers can only handle as many clients as there are available threads)
  • In modern non-blocking or asynchronous server architectures, the server doesn’t create a new thread for each client. Instead, it uses a small pool of threads (sometimes just one) that can handle many connections simultaneously. This is done by event-driven I/O (e.g., using select, poll, or epoll in Linux) or asynchronous frameworks that do not block on I/O operations.
•  In a blocking operation, the code execution is paused until the task is complete. The calling thread waits and does nothing else.
• In a non-blocking operation : the code execution doesn’t wait for the task to complete. Instead, it either polls or sets up a callback to check when the task is done. The thread can do other work in the meantime.
  • Asynchronous methods usually involve callbacks or promises (like CompletableFuture in Java). They allow your code to continue executing while the task finishes in the background.

 

Connection Pooling : 

Connection Reuse: Establishing a new HTTP connection for every request can be slow and resource-intensive. Connection pooling allows multiple requests to reuse the same TCP connection, significantly improving performance by reducing connection setup and teardown times.

 

 

 

Response Processing (Parsing JSON, XML)

Timeouts and Retries

Handling Slow External Services (Circuit Breakers)

Compression (Gzip)

Handling High-Throughput APIs

 

그 외 API 호출방식

1. RestTemplate

RestTemplate is one of the most straightforward and widely used classes in Spring for making synchronous HTTP requests to RESTful web services. While it's somewhat considered legacy with the introduction of newer, non-blocking clients like WebClient, it's still widely used, especially in legacy applications or simpler use cases.

 

2. WebClient

WebClient is part of the newer Spring WebFlux framework and is intended for making reactive (asynchronous non-blocking) web requests. It supports both traditional blocking I/O and non-blocking I/O through reactive streams. It is recommended for new developments when you need features like streaming, event-driven programming, or simply a more modern approach.

외부 Data 가공하기

JSON (JavaScript Object Notation) and XML (Extensible Markup Language) are two popular formats for data interchange between systems.  

 

JSON 데이터 가공 

 External Library : Gson

• • 1. toJson(Object src) : This method serializes the specified Java object into its equivalent JSON representation.
  • 2. fromJson(String json, Class<T> classOfT) : This method deserializes the specified JSON into an object of the specified class.
  • Gson의 fromJson class는 json의 내용물 양식과 동일한 형식의 class를 선언하고 해당 class를 선언하면 class의 attribute에 자동적으로 assign되고 결과적으로 class의 instance를 반환함. class는 사용자정의 class일 수도 있고 inbuilt class일 수도 있음.

// Java Servlet code
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    StringBuilder buffer = new StringBuilder();
    String line;
    while ((line = request.getReader().readLine()) != null) {
        buffer.append(line);
    }
    String jsonData = buffer.toString();
    JSONObject json = new JSONObject(jsonData);
    String userData = json.getString("data");

    // Process the data received...
    JSONObject responseJson = new JSONObject();
    responseJson.put("message", "Data processed successfully: " + userData);

    response.setContentType("application/json");
    PrintWriter out = response.getWriter();
    out.print(responseJson.toString());
    out.flush();
}

 

 

XML 데이터 가공 

 

JackSon Library 도입

 Json뿐만아니라 XML타입에도 호환이되며 성능이 준수하고 기능이 다양하여 여러 기업에서 선호되는 라이브러리임.

1. 의존성 도입하기 : implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.14.1'

2. XmlMapper을 생성하기, Spring에서는 싱글 instance로 진행하기 위해서 Bean생성하기

- DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES: When set to false, it allows the mapper to ignore unknown properties in the XML data during deserialization. If your XML contains fields that are not mapped to Java fields, it will skip those fields instead of throwing an error.

@Configuration
public class Config {

  @Bean
  public static XmlMapper xmlMapper() {
    XmlMapper xmlMapper = new XmlMapper();
    xmlMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
    return xmlMapper;
  }

 

2. XML데이터를 있는 그대로 옮길 JavaBean생성하기, List형태인 경우 List element도 JavaBean으로 생성하기

 

- XML Deserialization / readValue: This method is part of Jackson's XmlMapper (which extends ObjectMapper) and is used to convert a string (or other input) containing XML data into a Java object. In this case, the XML data is mapped to the HeritageApiResult class.

  try {
    log.info("Successfully parsed XML to HeritageApiResult");
    return xmlMapper.readValue(xmlResponse, HeritageApiResult.class);
  } catch (JsonProcessingException e) {
    log.error(e.getMessage());
    throw new CustomExcpetion(ErrorCode.EXTERNALAPI_NOT_FOUND, "Heritage API Not Found");
  }
}

 

3. Java Bean에 Getter와 Setter 생성하기, field를 annotation으로 mapping 시키기

 

Jackson은 자동적으로 Class와 Xml데이터 포맷을 자동적으로 맵핑하나 Class나 field 이름이 다르면 xmlProperty annotation으로 강제적으로 맵핑해줘야함.

 

@JacksonXmlRootElement(localName = "root_element"): map the root element of the XML document to your Java class.

@JacksonXmlProperty(localName = "element_name"): map individual XML elements to fields in your Java class when the names differ between the XML and the Java class.

@JacksonXmlElementWrapper(useWrapping = false) : 일반적으로 collection이 동일한 명의 item들이<item><> , <item><> 같이  나열되면  <items><>처럼 감싸주는 wrapper가 존재하는데, 본 xml에는 그런 존재가 없다는 것을 명시적으로 Jackson에게 알려주는 기능을 함.

@JacksonXmlRootElement(localName = "result")
public class HeritageApiResult {

  @JacksonXmlElementWrapper(useWrapping = false)
  @JacksonXmlProperty(localName = "item")
  List<HeritageApiItem> heritageApiItemList;

}

public class HeritageApiItem {

  @JacksonXmlProperty(localName = "ccbaCpno")
  private String heritageId; //
  @JacksonXmlProperty(localName = "ccbaMnm1")
  private String heritageName;
  private Double longitude;
  private Double latitude;
  @JacksonXmlProperty(localName = "ccmaName")
  private String heritageGrade;

  //ccbaKdcd, ccbaAsno, ccbaCtcd는 다른 APi 호출을 위해 필요한 para값
  private String ccbaKdcd;
  private String ccbaAsno;
  private String ccbaCtcd;

}

 

외부API사이트

• 공공데이터포탈 www.data.go.kr
•